Robust Physical-World Attacks on Deep Learning Visual Classification

By Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, Dawn Song. 2018

2,198 citations recorded by the source corpus.

Recent studies show that the state-of-the-art deep neural networks (DNNs) are vulnerable to adversarial examples, resulting from small-magnitude perturbations added to the input. Given that that emerging physical systems are using DNNs in safety-critical situations, adversarial examples could mislead these systems and cause dangerous situations. Therefore, understanding adversarial examples in the physical world is an important step towards developing resilient learning algorithms. We propose a general attack algorithm, Robust Physical Perturbations (RP2), to generate robust visual adversarial perturbations under different physical conditions. Using the real-world case of road sign classification, we show that adversarial examples generated using RP2 achieve high targeted misclassification rates against standard-architecture road sign classifiers in the physical world under various environmental conditions, including viewpoints. Due to the current lack of a standardized testing method, we propose a two-stage evaluation methodology for robust physical adversarial examples consisting of lab and field tests. Using this methodology, we evaluate the efficacy of physical adversarial manipulations on real objects. With a perturbation in the form of only black and white stickers, we attack a real stop sign, causing targeted misclassification in 100% of the images obtained in lab settings, and in 84.8% of the captured video frames obtained on a moving vehicle (field test) for the target classifier.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Molecular Biology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Molecular Biology

In Safety, Robustness & Interpretability

Citations in the corpus

Wasserstein Adversarial Examples via Projected Sinkhorn Iterations (2019, background): “Some examples include wearing adversarial 3D printed glasses to fool facial recognition (Sharif et al., 2017), the use of adversarial grafﬁti to attack trafﬁc sign classiﬁcation (Eykholt et al., 2018), and printing adversarial textures on objects to attack image classiﬁers (Athalye et al., 2018b).”
STRIP: a defence against trojan attacks on deep neural networks (2019, background): “As another attack example in [8], [13], an input-agnostic trigger ar X iv :1 90 2.”
Towards Robust Neural Networks via Random Self-ensemble (2017, background): “Deep neural networks have demonstrated their success in many machine learning and computer vision applications, including image classiication [14, 7, 35, 9, 34], object recognition [30] and image captioning [38].”
Constructing Unrestricted Adversarial Examples with Generative Models (2018, background): “This has raised concerns for safety-critical AI applications because, for example, attackers could use them to mislead autonomous driving vehicles [3, 4, 5] or hijack voice controlled intelligent agents [6, 7, 8].”
Robust fine-tuning of zero-shot models (2021, background): “Previous work has studied model behavior under synthetic [2, 23, 29, 36, 63, 97] and natural distribution shift [4, 35, 38, 47, 98].”
Machine learning and deep learning (2021, background): “They can be a threat to high-stake applications, for example, in terms of perturbations of street signs for autonomous driving (Eykholt et al. 2018).”
Regional Homogeneity: Towards Learning Transferable Universal Adversarial Perturbations Against Defenses (2019, background): “The variants of adversarial attacks [3, 9, 12, 20, 21, 30, 32, 33, 53, 56, 63, 64, 80] cast a security threat when deploying machine learning systems.”
CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances (2020, background): “, medical diagnosis [4] or autonomous driving [12]), and has a broad range of industrial applications (e.”
Backtracking gradient descent method for general $C^1$ functions, with applications to Deep Learning (2018, background): “…in the stochastic case (for example, for the common practice of using mini-batches), which will need to be thoroughly addressed if one wants to have a ﬁrm theoretical foundation to justify the use of GD in Deep Learning (in particular, in the rise of adversarial images, sounds and texts [36, 13]).”
Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (2020, background): “showed that placing small stickers on traffic signs (here: stop signs) can induce a misclassification rate of 100% in lab settings and 85% in a field test where video frames captured from a moving vehicle [172].”
Robust Feature-Level Adversaries are Interpretability Tools (2021, background): “(2018) [15] 4 7 4 4 7 7 7 Athalye et al.”
Is Robustness the Cost of Accuracy? - A Comprehensive Study on the Robustness of 18 Deep Image Classification Models (2018, background): “For example, [12] has shown that a subtly-modified physical Stop sign can be misidentified by a real-time object recognition system as a Speed Limit sign.”
Physically Realizable Adversarial Examples for LiDAR Object Detection (2020, background): “[13] proposes to train the attack with different view angles and distances to make it robust.”
Towards Stable and Efficient Training of Verifiably Robust Neural Networks (2019, background): “The robustness of DNNs under adversarial examples is well-studied from both attack (crafting powerful adversarial examples) and defence (making the model more robust) perspectives (Athalye et al., 2018; Carlini & Wagner, 2017a;b; Goodfellow et al., 2015; Madry et al., 2018; Papernot et al., 2016; Xi”