Detecting Adversarial Samples from Artifacts
By Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, Andrew B. Gardner. 2017
984 citations recorded by the source corpus.
Deep neural networks (DNNs) are powerful nonlinear architectures that are known to be robust to random perturbations of the input. However, these models are vulnerable to adversarial perturbations--small input changes crafted explicitly to fool the model. In this paper, we ask whether a DNN can distinguish adversarial samples from their normal and noisy counterparts. We investigate model confidence on adversarial samples by looking at Bayesian uncertainty estimates, available in dropout neural networks, and by performing density estimation in the subspace of deep features learned by the model. The result is a method for implicit adversarial detection that is oblivious to the attack algorithm. We evaluate this method on a variety of standard datasets including MNIST and CIFAR-10 and show that it generalizes well across different architectures and attacks. Our findings report that 85-93% ROC-AUC can be achieved on a number of standard classification tasks with a negative class that consists of both normal and noisy samples.
Subfields: Artificial Intelligence, Genetics, Safety, Robustness & Interpretability, Anomaly & Outlier Detection
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Bernt Schiele
- Percy Liang
- Yoshua Bengio
- Nicholas Carlini
- Quoc V. Le
- Mohit Bansal
- Sergey Levine
- Jun Zhu
In Genetics
- Fabio Roli
- Battista Biggio
- Giorgio Fumera
- Nicholas Carlini
- Ajmal Saeed Mian
- Naveed Akhtar
- Matthias Hein
- Dawn Song
In Safety, Robustness & Interpretability
- Nicholas Carlini
- Matthias Hein
- Cho-Jui Hsieh
- Nicolas Papernot
- Bo Li
- Huan Zhang
- Dawn Song
- Bernt Schiele
In Anomaly & Outlier Detection
- Nicholas Carlini
- Nicolas Papernot
- Patrick Drew McDaniel
- Matthias Hein
- Dawn Song
- Lewis Smith
- Ling Shao
- Michael Backes
Citations in the corpus
- SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, background): “Feinman et al. [11] also trains a classifier to detect adversarial perturbational inputs based on the neural network features, while Gong et al. [13] introduces a classifier trained to detect adversarially-perturbed images.”
- A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations (2017, background): “[8] Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner.”
- ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models (2017, background): “readers can refer to recent works in [10, 13, 18, 28, 42, 43] and references therein for details.”
- Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks (2017, methodology): “borrowed an idea from dropout [14] and designed a detection technique they called Bayesian neural network uncertainty [9].”
- On the (Statistical) Detection of Adversarial Examples (2017, methodology): “k. 9 RELATED WORK Other approaches to detect malicious data points by statistical means have been proposed. However, they all depend on some of the internal activations of deep neural networks models [20, 30, 32]. Hence, these approaches only apply to the specific classifier studied. In contrast, we”
- Towards Robust Neural Networks via Random Self-ensemble (2017, methodology): “In addition to changing the network structure, there are other methods [26, 15, 6, 8] “detecting” the adversarial examples, which are not in the scope of our paper.”
- Mitigating adversarial effects through randomization (2017, background): “Recent works (Papernot et al., 2016b; Kurakin et al., 2017; Tram`er et al., 2017; Metzen et al., 2017; Feinman et al., 2017; Meng & Chen, 2017) are making progress on this line of research.”
- PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples (2017, methodology): “Feinman et al. (2017) also presents a detection method based on kernel density estimation and Bayesian neural network uncertainty.”
- Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods (2017, methodology): “We study ten detection schemes proposed in seven papers over the last year [4, 11, 12, 15, 18, 19, 24], and compare their efficacy with the other defenses in a consistent manner.”
- Is Deep Learning Safe for Robot Vision? Adversarial Examples Against the iCub Humanoid (2017, background): “Note that this is a general issue for deep networks, not only specific to ImageNet [26, 9, 25, 10, 19, 20].”
- Efficient Defenses Against Adversarial Attacks (2017, methodology): “To this end, a certain number of directions have been explored, such as performing statistical tests [13], using an additional model for detection [11, 26] or applying dropout [36] at test time [9].”
- An overview of deep learning in medical imaging focusing on MRI (2018, background): “In addition to producing valuable measures that function as uncertainty measures [65,355,356], these techniques can also lessen deep neural networks susceptibility to adversarial attacks [354,357].”
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks (2018, methodology): “Suppose that a pre-trained softmax neural classifier is given: , where w c and b c are the weight and the bias of the soft-max classifier for class c , and f ( · ) denotes the output of the penultimate layer of DNNs.”
- Motivating the Rules of the Game for Adversarial Example Research (2018, background): “[69] so long as the attacker must go first.”
- Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong (2017, methodology): “examine the final hidden layer of a neural network and find that adversarial examples are separable from the original images by training a density estimate using Gaussian kernels [5].”
- Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “[86] proposed to detect adversarial perturbations by harnessing uncertainty estimates (of dropout neural networks) and performing density estimation in the feature space of neural networks.”
- EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples (2017, methodology): “Detection methods: Detection methods utilize statistical tests to differentiate adversarial from benign examples (Feinman et al. 2017; Grosse et al. 2017; Lu, Issaranon, and Forsyth 2017; Xu, Evans, and Qi 2017).”
- Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality (2018, methodology): “We deliberately did not tune the DNNs, as their performance was close to the state-of-the-art and could thus be considered sufficient for use in an adversarial study (Feinman et al., 2017).”