Towards Evaluating the Robustness of Neural Networks

By Nicholas Carlini, David A. Wagner. 2016

10,045 citations recorded by the source corpus.

Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to adversarial examples: given an input x and any target classification t, it is possible to find a new input x' that is similar to x but classified as t. This makes it difficult to apply neural networks in security-critical areas. Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from 95% to 0.5%.In this paper, we demonstrate that defensive distillation does not significantly increase the robustness of neural networks by introducing three new attack algorithms that are successful on both distilled and undistilled neural networks with 100% probability. Our attacks are tailored to three distance metrics used previously in the literature, and when compared to previous adversarial example generation algorithms, our attacks are often much more effective (and never worse). Furthermore, we propose using high-confidence adversarial examples in a simple transferability test we show can also be used to break defensive distillation. We hope our attacks will be used as a benchmark in future defense attempts to create neural networks that resist adversarial examples.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Percy Liang
  2. Yoshua Bengio
  3. Mohit Bansal
  4. Quoc V. Le
  5. Sergey Levine
  6. Bernt Schiele
  7. Jianfeng Gao
  8. Ruslan Salakhutdinov

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Ming-Hsuan Yang
  5. Bernt Schiele
  6. Xiaogang Wang
  7. Alan L. Yuille
  8. Jitendra Malik

In Safety, Robustness & Interpretability

  1. Nicholas Carlini
  2. Matthias Hein
  3. Nicolas Papernot
  4. Cho-Jui Hsieh
  5. Huan Zhang
  6. Florian Tramèr
  7. Aleksander Madry
  8. Bo Li

Citations in the corpus