Early Methods for Detecting Adversarial Images

By Dan Hendrycks, Kevin Gimpel. 2016

250 citations recorded by the source corpus.

Many machine learning classifiers are vulnerable to adversarial perturbations. An adversarial perturbation modifies an input to change a classifier's prediction without causing the input to seem substantially different to human perception. We deploy three methods to detect adversarial images. Adversaries trying to bypass our detectors must make the adversarial image less pathological or they will fail trying. Our best detection method reveals that adversarial images place abnormal emphasis on the lower-ranked principal components from PCA. Other detectors and a colorful saliency map are in an appendix.

Subfields: Artificial Intelligence, Biophysics, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Safety, Robustness & Interpretability

Citations in the corpus

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (2019, background): “…has focused on the important challenges of robustness to adversarial examples (Szegedy et al., 2014; Carlini & Wagner, 2017; 2016), unknown unknowns (Hendrycks et al., 2019; Hendrycks & Gimpel, 2017b; Liu et al., 2018), and model or data poisoning (Steinhardt et al., 2017; Hendrycks et al., 2018).”
Certified Robustness to Adversarial Examples with Differential Privacy (2018, methodology): “These defenses include model distialltion [44], automated detection of adversarial examples [22], [41], [40], application of different input transformations [27], [10], randomization [21], [11], and application of generative models [49], [25], [60].”
SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems (2018, neutral): “Grosse et al. [15] uses statistical techniques to distinguish adversarial-perturbations outputs, while Hendrycks et al. [18] uses PCA to visualize differences in perturbed images.”
Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification (2017, background): “One line of research [5, 6, 8, 11, 20, 24, 40] aim to detect adversarial examples, i.”
Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models (2018, neutral): “…2014; Goodfellow et al., 2015), (2) modifying the training procedure of the classiﬁer to reduce the magnitude of gradients, e.g., defensive distillation (Papernot et al., 2016d), and (3) attempting to remove the adversarial noise from the input samples (Hendrycks & Gimpel, 2017; Meng & Chen, 2017).”
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods (2017, methodology): “We study ten detection schemes proposed in seven papers over the last year [4, 11, 12, 15, 18, 19, 24], and compare their efficacy with the other defenses in a consistent manner.”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “However, Carlini and Wagner [77], [78] summarized most of these adversarial detecting methods (see [89], [109]–[113])”
Using Pre-Training Can Improve Model Robustness and Uncertainty (2019, methodology): “Nguyen & O’Connor (2015); Hendrycks & Gimpel (2017b) found that deep neural network classiﬁers display severe overcon-ﬁdence in their predictions, and that the problem becomes worse with increased representational capacity (Guo et al., 2017).”
Towards neural networks that provably know when they don't know (2019, methodology): “Only recently the detection of OOD inputs (Hendrycks & Gimpel, 2017a) has been tackled.”
Motivating the Rules of the Game for Adversarial Example Research (2018, methodology): “Whatever method the adversary uses to build adversarial examples ultimately deﬁnes some distribution of inputs, which could be easily detected statistically as in Hendrycks and Gimpel [41], Grosse et al. [64], and Feinman et al. [69] so long as the attacker must go ﬁrst.”
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong (2017, background): “wn that adversarial examples remain even when subject to the lossy channel of being photographed [17]. Developing effective defenses against adversarial examplesisanimportanttopic. Despitemanyattempts[27, 7,9,21,8,6,5,11],thereisnostrongdefenseagainstadversarial examples to date. Inthispaper,weaskth”
SentiNet: Detecting Physical Attacks Against Deep Learning Systems (2018, unknown): “Grosse et al. [19] uses statistical techniques to distinguish adversarial-perturbations outputs, while Hendrycks et al. [23] uses PCA to visualize differences in perturbed images.”
On the Connection between Differential Privacy and Adversarial Robustness in Machine Learning (2018, background): “Several other defenses [14, 31, 30] proposed detecting adversarial examples by leveraging their statistical differences to clean examples.”
Attack Strength vs. Detectability Dilemma in Adversarial Machine Learning (2018, background): “While there has been significant work into detecting adversarial examples [29]–[31], they have had limited success.”
A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability (2018, background): “For example, most defences against attacks in the white-box setting, including [Papernot et al., 2016a, Metzen et al., 2017, Hendrycks and Gimpel, 2016, Meng and Chen, 2017a], have been demonstrated to be vulnerable to e.g., iterative optimisation-based attacks [Carlini and Wagner, 2017b,Carlini…”
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review (2019, methodology): “The property of transferability is also utilized by defending methods as in (Hendrycks & Gimpel, 2016): since the adversarial examples for model A are also likely to be adversarial for model B, adversarial training using adversarial examples from B will help defend A.”
Machine Learning as an Adversarial Service: Learning Black-Box Adversarial Examples (2017, methodology): “Hendrycks & Gimpel’s [10] PCA whitening method Source image Perturbed source image Adversarial image predicted class: 8 At ta ck w ei gh t 1 0 Adversarial pixels Adversarial image predicted class: 8 At ta ck w ei gh t 2 0 Adversarial pixels Adversarial image predicted class: 0 At ta ck w ei gh t 4 0”
Learning Universal Adversarial Perturbations with Generative Models (2017, background): “t background information increasing the impact of our attacks. 5) In Section 7, we show that our attack is capable of adapting and weakening the robustness of two popular adversarial example defenses [10], [39]. 6) In Section 8, we show that adversarial examples crafted by the attack can successfull”