Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

By Anish Athalye, Nicholas Carlini, David A. Wagner. 2018

3,560 citations recorded by the source corpus.

We identify obfuscated gradients, a kind of gradient masking, as a phenomenon that leads to a false sense of security in defenses against adversarial examples. While defenses that cause obfuscated gradients appear to defeat iterative optimization-based attacks, we find defenses relying on this effect can be circumvented. We describe characteristic behaviors of defenses exhibiting the effect, and for each of the three types of obfuscated gradients we discover, we develop attack techniques to overcome it. In a case study, examining non-certified white-box-secure defenses at ICLR 2018, we find obfuscated gradients are a common occurrence, with 7 of 9 defenses relying on obfuscated gradients. Our new attacks successfully circumvent 6 completely, and 1 partially, in the original threat model each paper considers.

Subfields: Artificial Intelligence, Hardware and Architecture, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Percy Liang
  2. Yoshua Bengio
  3. Quoc V. Le
  4. Mohit Bansal
  5. Sergey Levine
  6. Jianfeng Gao
  7. Ruslan Salakhutdinov
  8. Bernt Schiele

In Hardware and Architecture

  1. Matthias Hein
  2. Nicholas Carlini
  3. Francesco Croce
  4. J. Kolter
  5. Cho-Jui Hsieh
  6. Huan Zhang
  7. Tom Goldstein
  8. Florian Tramèr

In Safety, Robustness & Interpretability

  1. Matthias Hein
  2. Nicholas Carlini
  3. Aleksander Madry
  4. Florian Tramèr
  5. Nicolas Papernot
  6. Huan Zhang
  7. Cho-Jui Hsieh
  8. Francesco Croce

Citations in the corpus