Are adversarial examples inevitable?

By Ali Shafahi, W. Ronny Huang, Christoph Studer, Soheil Feizi, Tom Goldstein. 2018

305 citations recorded by the source corpus.

A wide range of defenses have been proposed to harden neural networks against adversarial attacks. However, a pattern has emerged in which the majority of adversarial defenses are quickly broken by new attacks. Given the lack of success at generating robust defenses, we are led to ask a fundamental question: Are adversarial attacks inevitable? This paper analyzes adversarial examples from a theoretical perspective, and identifies fundamental bounds on the susceptibility of a classifier to adversarial attacks. We show that, for certain classes of problems, adversarial examples are inescapable. Using experiments, we explore the implications of theoretical guarantees for real-world problems and discuss how factors such as dimensionality and image complexity limit a classifier's robustness against adversarial examples.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Feature Purification: How Adversarial Training Performs Robust Deep Learning (2020, background): “In particular, we found that the following fundamental questions remain largely unaddressed: Questions Why do adversarial examples exist when we train the neural networks using the original training data set? How can adversarial training further “robustify” the trained neural networks against these ”
Adversarial Training and Robustness for Multiple Perturbations (2019, background): “1 We follow a line of works that study distributions for which adversarial examples exist unconditionally [36, 18, 30, 10, 12, 23].”
Adversarial Training for Free! (2019, background): “Another popular defense strategy is to regularize the training loss using label smoothing, logit squeezing, or a Jacobian regularization [Shafahi et al., 2019a, Mosbach et al., 2018, Ross and Doshi-Velez, 2018, Hein and Andriushchenko, 2017, Jakubovitz and Giryes, 2018, Yu et al., 2018].”
Certified Adversarial Robustness via Randomized Smoothing (2019, background): “Recent theoretical work has taken steps towards addressing this question (Mahloujifar et al., 2019; Gilmer et al., 2018b; Fawzi et al., 2018; Tsipras et al., 2019; Schmidt et al., 2018; Shafahi et al., 2019; Dohma- tob, 2018; Bubeck et al., 2018b).”
Adversarial Policies: Attacking Deep Reinforcement Learning (2019, background): “It is well-established that classiﬁers are more vulnerable to adversarial examples on high-dimensional inputs (Gilmer et al., 2018b; Khoury and Hadﬁeld-Menell, 2018; Shafahi et al., 2019).”
Generating synthetic data in finance: opportunities, challenges and pitfalls (2020, background): “There is theoretical justification for this being the rule rather than the exception, see for instance [47].”
Adversarial frontier stitching for remote neural network watermarking (2017, methodology): “nd MLP, while there are few extraction failures from the attacked IRNN architecture. This experiment is thus consistent with recent arguments about the general difﬁculty to defend against adversaries [34,31]. Conclusion on the attacks of watermarks We highlight that in all but two tries, the waterma”
Simple Black-box Adversarial Attacks (2019, background): “Although often misunderstood as a property of neural networks (Szegedy et al., 2014), the vulnerability towards adversarial examples is likely an inevitability of classiﬁers in high-dimensional spaces with most data distributions (Fawzi et al., 2018; Shafahi et al., 2018).”
Adversarial Robustness May Be at Odds With Simplicity (2019, background): “First, there are several works arguing that robust classifiers simply may not exist [7, 9, 4].”
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review (2019, methodology): “close to decision boundary of a natural trained model, which may be because the decision boundary is too flat[92], too curved[93], or inflexible[94].”
Limitations of adversarial robustness: strong No Free Lunch Theorem (2018, background): “Results for the the hyper-cube image model considered in (Shafahi et al., 2018) can be recovered by noting that the uniform measure on [0, 1] is the pushforward T#γp of the standard Gaussian on R, where T : R → [0, 1] is (2π)−1/2-Lipschitz map defined by T (z1, .”
Anomalous Instance Detection in Deep Learning: A Survey (2020, background): “Therefore, the development of a coherent theory and methodology that guides practical design for anomaly detection in DL-based systems [116], and fundamental characterizations of the existence of adversarial examples [117] is of utmost importance.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “We also find a similar but broader claim that adversarial examples are inevitable for certain types of problems in [312].”
Universal Adversarial Training (2018, background): “…per-instance perturbations as small as (cid:15) = 2 are often enough to considerably degrade the performance of conventionally trained classiﬁers possibly due to the fact that ImageNet is a complex dataset and is fundamentally susceptible to per-instance perturbations (Shafahi et al., 2018).”
A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance (2019, methodology): “Typical examples of such results are Shafahi et al. (2018) which shows that any measurable predictor on the unit hypercube [0 , 1] n will have an adversarial example if perturbed strongly enough, and Fawzi et al. (2018) which shows that any predictor on a data distribution supported on a…”
A Survey On Universal Adversarial Attack (2021, background): “…which motivates numerous works attempting to explain its existence from a wide range of perspectives, such as the local linearity of DNNs [Goodfellow et al., 2015], input high-dimensionality [Shafahi et al., 2019a] and over-fitting [Schmidt et al., 2018], and noise disturbance [Fawzi et al., 2016].”