On the (Statistical) Detection of Adversarial Examples

By Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, Patrick Drew McDaniel. 2017

768 citations recorded by the source corpus.

Machine Learning (ML) models are applied in a variety of tasks such as network intrusion detection or Malware classification. Yet, these models are vulnerable to a class of malicious inputs known as adversarial examples. These are slightly perturbed inputs that are classified incorrectly by the ML model. The mitigation of these adversarial inputs remains an open problem. As a step towards understanding adversarial examples, we show that they are not drawn from the same distribution than the original data, and can thus be detected using statistical tests. Using thus knowledge, we introduce a complimentary approach to identify specific inputs that are adversarial. Specifically, we augment our ML model with an additional output, in which the model is trained to classify all adversarial inputs. We evaluate our approach on multiple adversarial example crafting methods (including the fast gradient sign and saliency map methods) with several datasets. The statistical test flags sample sets containing adversarial inputs confidently at sample sizes between 10 and 100 data points. Furthermore, our augmented model either detects adversarial examples as outliers with high accuracy (> 80%) or increases the adversary's cost - the perturbation added - by more than 150%. In this way, we show that statistical properties of adversarial examples are essential to their detection.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Local Model Poisoning Attacks to Byzantine-Robust Federated Learning (2019, neutral): “, [10,25,26,38,41,48,64]) were explored to mitigate them.”
SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, neutral): “Grosse et al. [15] uses statistical techniques to distinguish adversarial-perturbations outputs, while Hendrycks et al. [18] uses PCA to visualize differences in perturbed images.”
A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations (2017, background): “[11] Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel.”
ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models (2017, background): “readers can refer to recent works in [10, 13, 18, 28, 42, 43] and references therein for details.”
Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification (2017, background): “We note that the second limitation also applies to other approaches [4, 13] that aim to detect adversarial examples.”
Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks (2017, methodology): “[29]) that adds a new “adversarial” class in the last layer of the DNN model [11].”
The Robust Manifold Defense: Adversarial Training using Generative Models (2017, background): “This phenomenon is receiving a tremendous amount of recent attention (e.g. [31, 25, 48, 19, 22, 46] and references therein) for two good reasons: First a classiﬁer that can be easily fooled by non-perceivable noise poses a security threat in any real deployment.”
Towards Robust Neural Networks via Random Self-ensemble (2017, methodology): “In addition to changing the network structure, there are other methods [26, 15, 6, 8] “detecting” the adversarial examples, which are not in the scope of our paper.”
PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples (2017, background): “Most recent work on detecting adversarial examples focuses on adding an outlier class detection module to the classiﬁer, such as Grosse et al. (2017), Gong et al. (2017) and Metzen et al. (2017).”
MagNet: A Two-Pronged Defense against Adversarial Examples (2017, background): “is to detect adversarial examples with hand-crafted statistical features [7] or separate classification networks [20].”
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods (2017, methodology): “We study ten detection schemes proposed in seven papers over the last year [4, 11, 12, 15, 18, 19, 24], and compare their efficacy with the other defenses in a consistent manner.”
Efficient Defenses Against Adversarial Attacks (2017, methodology): “To this end, a certain number of directions have been explored, such as performing statistical tests [13], using an additional model for detection [11, 26] or applying dropout [36] at test time [9].”
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong (2017, background): “Despite many attempts [27, 7, 9, 21, 8, 6, 5, 11], there is no strong defense against adversarial examples to date.”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “[57] proposed to augment the potentially targeted neural network model with an additional class in which the model is trained to classify all the adversarial examples.”
EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples (2017, methodology): “Detection methods: Detection methods utilize statistical tests to differentiate adversarial from benign examples (Feinman et al. 2017; Grosse et al. 2017; Lu, Issaranon, and Forsyth 2017; Xu, Evans, and Qi 2017).”
Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality (2018, methodology): “(Li & Li, 2016), an augmented neural network detector based on statistical measures, a logistic regression detector based on KD, and Bayesian Uncertainty (BU) features (Grosse et al., 2017) are a few such works.”