SafetyNet: Detecting and Rejecting Adversarial Examples Robustly

By Jiajun Lu, Theerasit Issaranon, D.A. Forsyth. 2017

403 citations recorded by the source corpus.

We describe a method to produce a network where current methods such as DeepFool have great difficulty producing adversarial samples. Our construction suggests some insights into how deep networks work. We provide a reasonable analyses that our construction is difficult to defeat, and show experimentally that our method is hard to defeat with both Type I and Type II attacks using several standard networks and datasets. This SafetyNet architecture is used to an important and novel application SceneProof, which can reliably detect whether an image is a picture of a real scene or not. SceneProof applies to images captured with depth maps (RGBD images) and checks if a pair of image and depth map is consistent. It relies on the relative difficulty of producing naturalistic depth maps for images in post processing. We demonstrate that our SafetyNet is robust to adversarial examples built from currently known attacking approaches.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Safety, Robustness & Interpretability

Citations in the corpus

SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, background): “Safetynets [22] is designed to detect adversarial-noise based attacks and exploits the different activations adversarial perturbations produce to train a SVM classiﬁer.”
NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles (2017, background): “In the past few years, researchers tried to explain why neural nets are susceptible to such examples despite their impressive success on random test datasets [6, 5, 2], suggested new methods to generate adversarial examples and measure NN robustness against them [4, 21, 16, 9, 17, 19, 13], and propo”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, methodology): “Many research projects tried to detect adversarial examples in the testing stage [23], [89], [109]–[114], [116].”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “[66] later showed that this approach is again vulnerable to counter-counter measures.”
EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples (2017, neutral): “(Feinman et al. 2017; Grosse et al. 2017; Lu, Issaranon, and Forsyth 2017; Xu, Evans, and Qi 2017).”
Improving Network Robustness against Adversarial Attacks with Compact Convolution (2017, background): “Methods based on adversarial detection [14, 15, 7, 5] attempt to detect an adversarial sample before passing it through the network for inference.”
Empirical Study of the Topology and Geometry of Deep Networks (2018, methodology): “Interestingly enough, [22, 21] report that the regime where perturbations are very small (like those produced by [18]) are the hardest to detect; we therefore foresee that this geometric approach will be used along with other detection approaches, as it provides very accurate detection in a distinct”
Standard detectors aren't (currently) fooled by physical adversarial stop signs (2017, background): “Current procedures to build adversarial examples for deep networks appear to subvert the feature construction implemented by the network to produce odd patterns of activation in late stage RELU’s; this can be exploited to build one form of defence [10].”
SentiNet: Detecting Physical Attacks Against Deep Learning Systems (2018, background): “Safetynets [31] is designed to detect adversarial-noise based attacks and exploits the different activations adversarial perturbations produce to train a SVM classiﬁer.”
Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems (2019, methodology): “In addition, we also investigate the use of deep features (denoted by “DFeat”) or quantized deep features (denoted by “QFeat”) (Lu et al., 2017) for adversarial detection.”
With Friends Like These, Who Needs Adversaries? (2018, methodology): “A variety of training-based [8, 18, 19, 20] methods are proposed to address the premise of the preceding analyses.”
Secure and Robust Machine Learning for Healthcare: A Survey (2020, background): “binary classifier is trained to distinguish between the adversarial and original samples that can be regarded as the detector model [182], [183].”
Few-Shot Open-Set Recognition Using Meta-Learning (2020, background): “images from other domains or adversarial attacks [11]), realistic classification [19], where it rejects examples that it deems too hard to classify, or open-set recognition [1], where the examples to reject are from novel classes, unseen during training.”
Adversarial Examples on Object Recognition (2020, methodology): “They discover that rotation equivariant networks are signiﬁcantly more robust to attacks based on small translations and rotations, but marginally robust against attacks based on local geometric distortions.”
Anomalous Instance Detection in Deep Learning: A Survey (2020, background): “In [71] SafetyNet is proposed which consists of the original classifier, and an adversary detector which looks at the internal state of the later layers in the original classifier.”
Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization (2018, background): “Some of the strategies aim at detecting whether an input image is adversarial or not (e.g., [9,10,11,12,13,14]).”