Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods

By Nicholas Carlini, David A. Wagner. 2017

1,995 citations recorded by the source corpus.

Neural networks are known to be vulnerable to adversarial examples: inputs that are close to natural inputs but classified incorrectly. In order to better understand the space of adversarial examples, we survey ten recent proposals that are designed for detection and compare their efficacy. We show that all can be defeated by constructing new loss functions. We conclude that adversarial examples are significantly harder to detect than previously appreciated, and the properties believed to be intrinsic to adversarial examples are in fact not. Finally, we propose several simple guidelines for evaluating future proposed defenses.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Natural Adversarial Examples (2019, background): “Aside from the known difﬁculties in evaluating (cid:96) p robustness [7, 8], Gilmer et al. [14] point out that (cid:96) p adversarial examples assume an unrealistic threat model because attackers are often free to choose any desired input.”
Adversarial Examples Are Not Bugs, They Are Features (2019, background): “…view, it is natural to treat adversarial robustness as a goal that can be disentangled and pursued independently from maximizing accuracy [Mad+18; SHS19; Sug+19], either through improved standard regularization methods [TG16] or pre/post-processing of network inputs/outputs [Ues+18; CW17a; He+17].”
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (2019, background): “Algorithms have been developed that search for the smallest additive distortions in RGB space that are sufﬁcient to confuse a classiﬁer (Carlini et al., 2017).”
Towards Deep Learning Models Resistant to Adversarial Attacks (2017, background): “Examples include defensive distillation [24, 6], feature squeezing [31, 14], and several other adversarial example detection approaches [5].”
Certified Robustness to Adversarial Examples with Differential Privacy (2018, background): “However, all these best-effort defenses have been broken, sometimes even as soon as a few months after their publication [7], [6], [2].”
Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning (2017, critical): “This is witnessed by the work by Carlini and Wagner [15, 79], who exploited a similar idea to show that several recent defenses proposed against minimally-perturbed adversarial examples are vulnerable to high-conﬁdence ones, using a stronger attack similar to those proposed in our earlier work, and…”
Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, background): “While some defense methods have been developed, more attack methods have also emerged [8], [9], [10], [11], and susceptibility to unintuitive errors remains a pervasive problem in DNNs.”
Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples (2018, methodology): “Much of what we describe below has been discussed in prior work (Carlini & Wagner, 2017a; Madry et al., 2018); we repeat these points here and offer our own perspective for completeness.”
A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations (2017, background): “[6] Nicholas Carlini and David Wagner.”
ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models (2017, background): “However, Carlini and Wagner recently demonstrated that their attack (C&W attack) can bypass 10 different detection algorithms designed for detecting adversarial examples [7], which challenges the fundamental assumption of detection-based approaches as the results suggest that the distributions of th”
Ensemble Adversarial Training: Attacks and Defenses (2017, methodology): “…the accuracy of linear approximations of the model’s loss is as an instance of a gradient-masking phenomenon (Papernot et al., 2016b), which affects other defensive techniques (Papernot et al., 2016c; Carlini & Wagner, 2017a; Nayebi & Ganguli, 2017; Brendel & Bethge, 2017; Athalye et al., 2018).”
Bypassing Backdoor Detection Algorithms in Deep Learning (2019, background): “In the case of adversarial examples, it has been shown that a large number of defense mechanisms can be bypassed by an adaptive attack, for the same weakness in their threat model [1], [6], [5].”
Attacking Graph-based Classification via Manipulating the Graph Structure (2019, neutral): “[13,27, 41,47]) and data poisoning attacks (e.g., [5,32,54]).”
When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks (2018, background): “These attacks appear to be very effective, and the defenses proposed against them are often bypassed in follow-on work [8].”
Mitigating Evasion Attacks to Deep Neural Networks via Region-based Classification (2017, background): “sting example to be benign or adversarial. An a−acker can strategically adjust its a−acks to evade both the original classi•er and the new classi•er to detect adversarial examples. Carlini and Wagner [3] demonstrated that, for such adaptive a−acks, some detectors are ine‡ective while some detectors ”
Certified Adversarial Robustness via Randomized Smoothing (2019, background): “In fact, most heuristic defenses proposed in the literature were later “broken” by stronger adversaries (Carlini & Wagner, 2017; Athalye et al., 2018; Uesato et al., 2018; Athalye & Carlini, 2018).”
Training verified learners with learned verifiers (2018, background): “It has been shown that the addition of small but carefully chosen deviations to the input, so-called adversarial perturbations, can cause the neural network to make incorrect predictions with very high confidence [4, 5, 12, 19, 26].”