NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles

By Jiajun Lu, Hussein Sibai, Evan Fabry, D.A. Forsyth. 2017

293 citations recorded by the source corpus.

It has been shown that most machine learning algorithms are susceptible to adversarial perturbations. Slightly perturbing an image in a carefully chosen direction in the image space may cause a trained neural network model to misclassify it. Recently, it was shown that physical adversarial examples exist: printing perturbed images then taking pictures of them would still result in misclassification. This raises security and safety concerns. However, these experiments ignore a crucial property of physical objects: the camera can view objects from different distances and at different angles. In this paper, we show experiments that suggest that current constructions of physical adversarial examples do not disrupt object detection from a moving platform. Instead, a trained neural network classifies most of the pictures taken from different distances and angles of a perturbed image correctly. We believe this is because the adversarial property of the perturbation is sensitive to the scale at which the perturbed picture is viewed, so (for example) an autonomous car will misclassify a stop sign only from a small range of distances. Our work raises an important question: can one construct examples that are adversarial for many or most viewing conditions? If so, the construction should offer very significant insights into the internal representation of patterns by deep networks. If not, there is a good prospect that adversarial examples can be reduced to a curiosity with little practical impact.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects (2018, background): “But pixel-wise changes are uncorrelated [28], so pixel-based attacks may not transfer well to the real world [24, 26] because there is an infinitesimal chance that such specifically crafted, uncorrelated pixels will be encountered in the vast physical space of camera, lighting, traffic, and weather ”
Certified Robustness to Adversarial Examples with Differential Privacy (2018, background): “Other work [36] claimed that adversarial examples are unlikely to fool machine learning (ML) models in the real-world due to the rotation and scaling introduced by even slightest camera movements.”
Wasserstein Adversarial Examples via Projected Sinkhorn Iterations (2019, background): “, 2016; Carlini & Wagner, 2017), realistic transformations seen by vehicles were thought to be safe until more robust adversarial examples were created (Lu et al., 2017; Athalye et al., 2018b), and many defenses submitted to ICLR 2018 were broken shortly after the review period finished (Athalye et ”
Provable defenses against adversarial examples via the convex outer adversarial polytope (2017, background): “There is no need to worry about adversarial examples under “realistic” settings of rotation and scaling [Lu et al., 2017] until there is [Athalye and Sutskever, 2017].”
Robust Physical-World Attacks on Deep Learning Models (2017, methodology): “performed experiments with physical adversarial examples of road sign images [16].”
Countering Adversarial Images using Input Transformations (2018, background): “For example, in the context of image classiﬁcation, adversarial perturbations can be partly removed via JPEG compression (Dziugaite et al., 2016) or image re-scaling (Lu et al., 2017).”
SafetyNet: Detecting and Rejecting Adversarial Examples Robustly (2017, background): “imagine a small physical modification that could reliably get a stop sign classified as a go faster sign [25, 16]).”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “[30] had previously claimed that adversarial examples are not a concern for object detection in Autonomous Vehicles because of the changing physical conditions in a moving car.”
Fast is better than free: Revisiting adversarial training (2020, background): “…can give a misleading sense of security, and the history of adversarial examples is littered with adversarial defenses (Papernot et al., 2016; Lu et al., 2017; Kannan et al., 2018; Tao et al., 2018) which were ultimately defeated by stronger attacks (Carlini & Wagner, 2016; 2017b; Athalye et…”
Semidefinite relaxations for certifying robustness to adversarial examples (2018, background): “A proposed defense based on transformations of test inputs [20] was broken in only five days [2].”
Adversarial Robustness Against the Union of Multiple Perturbation Models (2019, background): “Adversarial attacks were thought to be safe under realistic transformations (Lu et al., 2017) until the attack was augmented to be robust to them (Athalye et al.”
Retrieval-Augmented Convolutional Neural Networks for Improved Robustness against Adversarial Examples (2018, background): “Lu et al. (2017) suggest that trying various scaling of an image size could overcome adversarial attacks, as they seem to be sensitive to the scal-3 ing of objects.”
Standard detectors aren't (currently) fooled by physical adversarial stop signs (2017, background): “In earlier work, we showed that it was difficult to build physical examples that fooled a stop-sign detector [11].”
Traffic Signs in the Wild: Highlights from the IEEE Video and Image Processing Cup 2017 Student Competition [SP Competitions] (2018, background): “Recent studies [3, 4] showed that adversarial perturbations can degrade the performance of existing traffic sign recognition systems under specific conditions.”
On the Robustness of Semantic Segmentation Models to Adversarial Attacks (2017, unknown): “[46].”
Robust Physical-World Attacks on Deep Learning Visual Classification (2018, background): “Cyber-physical systems operate in noisy physical environments that can de-stroy perturbations created using current digital-only algorithms [19].”
CURE-OR: Challenging Unreal and Real Environments for Object Recognition (2018, background): “Moreover, adversarial examples investigated in [18, 19] are inherently different from realistic challenging scenarios because they are specifically tuned to bias the models.”
Exact Adversarial Attack to Image Captioning via Structured Output Learning With Latent Variables (2019, background): “, object detection and semantic segmentation, is also studied in [8, 22, 34] and [34, 8, 2], respectively.”
ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector (2018, background): “To the best of knowledge, our work presents the ﬁrst reproducible and robust targeted attack against Faster R-CNN [14].”
Feature-Guided Black-Box Safety Testing of Deep Neural Networks (2017, methodology): “A recent paper [21] argued that, under specific circumstances, there is no need to worry about adversarial examples because they are not invariant to changes in scale or angle in the physical domain.”