Adversarial examples in the physical world

By Alexey Kurakin, Ian J. Goodfellow, Samy Bengio. 2016

6,799 citations recorded by the source corpus.

Most existing machine learning classifiers are highly vulnerable to adversarial examples. An adversarial example is a sample of input data which has been modified very slightly in a way that is intended to cause a machine learning classifier to misclassify it. In many cases, these modifications can be so subtle that a human observer does not even notice the modification at all, yet the classifier still makes a mistake. Adversarial examples pose security concerns because they could be used to perform an attack on machine learning systems, even if the adversary has no access to the underlying model. Up to now, all previous work have assumed a threat model in which the adversary can feed data directly into the machine learning classifier. This is not always the case for systems operating in the physical world, for example those which are using signals from cameras and other sensors as an input. This paper shows that even in such physical world scenarios, machine learning systems are vulnerable to adversarial examples. We demonstrate this by feeding adversarial images obtained from cell-phone camera to an ImageNet Inception classifier and measuring the classification accuracy of the system. We find that a large fraction of adversarial examples are classified incorrectly even when perceived through the camera.

Subfields: Artificial Intelligence, Hardware and Architecture, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Percy Liang
  3. Mohit Bansal
  4. Quoc V. Le
  5. Sergey Levine
  6. Jianfeng Gao
  7. Bernt Schiele
  8. Ruslan Salakhutdinov

In Hardware and Architecture

  1. Nicholas Carlini
  2. Matthias Hein
  3. J. Kolter
  4. Eric Wong
  5. Cho-Jui Hsieh
  6. Bo Li
  7. Ian J. Goodfellow
  8. Huan Zhang

In Safety, Robustness & Interpretability

  1. Nicholas Carlini
  2. Matthias Hein
  3. Nicolas Papernot
  4. Bo Li
  5. Cho-Jui Hsieh
  6. Huan Zhang
  7. Dawn Song
  8. Bernt Schiele

In Signal Processing

  1. Nicholas Carlini
  2. Bo Li
  3. Yoshua Bengio
  4. Nicolas Papernot
  5. Bernt Schiele
  6. Matthias Hein
  7. Aleksander Madry
  8. Dawn Song

Citations in the corpus