ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector

By Shang-Tse Chen, Cory Cornelius, Jason Martin, Duen Horng Chau. ECML/PKDD, 2018

480 citations recorded by the source corpus.

Given the ability to directly manipulate image pixels in the digital input space, an adversary can easily generate imperceptible perturbations to fool a Deep Neural Network (DNN) image classifier, as demonstrated in prior work. In this work, we propose ShapeShifter, an attack that tackles the more challenging problem of crafting physical adversarial perturbations to fool image-based object detectors like Faster R-CNN. Attacking an object detector is more difficult than attacking an image classifier, as it needs to mislead the classification results in multiple bounding boxes with different scales. Extending the digital attack to the physical world adds another layer of difficulty, because it requires the perturbation to be robust enough to survive real-world distortions due to different viewing distances and angles, lighting conditions, and camera limitations. We show that the Expectation over Transformation technique, which was originally proposed to enhance the robustness of adversarial perturbations in image classification, can be successfully adapted to the object detection setting. ShapeShifter can generate adversarially perturbed stop signs that are consistently mis-detected by Faster R-CNN as other objects, posing a potential threat to autonomous vehicles and other safety-critical computer vision systems.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Hardware and Architecture

View Paper: https://www.semanticscholar.org/paper/e77fb2fa94916505bc8af5b56bc503d8bfac7fb9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Hardware and Architecture

Citations in the corpus

Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors (2019, background): “Stop signs attacks on object detectors are considered in [8,5].”
Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks (2021, methodology): “Multiple prior works have designed and evaluated adversarial attacks in the physical world [5]–[9], [57]–[60].”
Physical Adversarial Examples for Object Detectors (2018, background): “bject detector [12]. Their results suggested that it is very challenging to fool YOLO with physical adversarial examples. Concurrent to our work, Chen et al. attacked the Faster R-CNN object detector [4]. Their attack relies on generating adversarial poster perturbations that replace the road sign t”
Kornia: an Open Source Differentiable Computer Vision Library for PyTorch (2019, background): “There are series of works showing that CNNbased solution of classification [1], segmentation [4], object detection [11], and image retrieval [32] tasks are all prone to such attacks.”
Towards Adversarially Robust Object Detection (2019, methodology): “It is clear that some methods use classification loss [6, 32, 11, 57, 55] while other methods also incorporated localization loss [31, 23, 22].”
StegaStamp: Invisible Hyperlinks in Physical Photographs (2019, background): “Most relevant to our work are the demonstrations of adversarial examples in the physical world [4, 10, 15, 25, 28, 40, 41], where systems are made robust for imaging applications by modeling physically realistic perturbations (i.e., afﬁne image warping, additive noise, and JPEG compression).”