Adversarial vulnerability for any classifier

By Alhussein Fawzi, Hamza Fawzi, Omar Fawzi. 2018

273 citations recorded by the source corpus.

Despite achieving impressive performance, state-of-the-art classifiers remain highly vulnerable to small, imperceptible, adversarial perturbations. This vulnerability has proven empirically to be very intricate to address. In this paper, we study the phenomenon of adversarial perturbations under the assumption that the data is generated with a smooth generative model. We derive fundamental upper bounds on the robustness to perturbations of any classification function, and prove the existence of adversarial perturbations that transfer well across different classifiers with small risk. Our analysis of the robustness also provides insights onto key properties of generative models, such as their smoothness and dimensionality of latent space. We conclude with numerical experimental results showing that our bounds provide informative baselines to the maximal achievable robustness on several datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial Examples Are Not Bugs, They Are Features (2019, methodology): “An orthogonal line of work [Gil+18; FFF18; MDM18; Sha+19a], argues that the high dimensionality of the input space can present fundamental barriers on clas-siﬁer robustness.”
Adversarial Training and Robustness for Multiple Perturbations (2019, background): “We follow a line of works that study distributions for which adversarial examples exist unconditionally [41, 21, 33, 12, 14, 26].”
Certified Adversarial Robustness via Randomized Smoothing (2019, background): “Recent theoretical work has taken steps towards addressing this question (Mahloujifar et al., 2019; Gilmer et al., 2018b; Fawzi et al., 2018; Tsipras et al., 2019; Schmidt et al., 2018; Shafahi et al., 2019; Dohma- tob, 2018; Bubeck et al., 2018b).”
Theoretical evidence for adversarial robustness through randomization: the case of the Exponential family (2019, background): “g [42], or [15] denoted α-radius prediction-change risk of h w.”
Excessive Invariance Causes Adversarial Vulnerability (2018, background): “Some works (Song et al., 2018; Fawzi et al., 2018) consider unrestricted adversarial examples, which are closely related to invariance-based adversarial vulnerability.”
Robustness May Be at Odds with Accuracy (2018, background): “Due to the large body of related work, we will only focus on the most relevant studies here and defer the full discussion to Appendix F. Fawzi et al. (2018b) prove upper bounds on the robust of classiﬁers and exhibit a standard vs. robust accuracy trade-oﬀ for a speciﬁc classiﬁer families on a…”
Adversarially Robust Generalization Requires More Data (2018, background): “Finally, Fawzi, Fawzi, and Fawzi [16] utilize the notion of a latent space for the data distribution in order to establish lower bounds that apply to any classifier.”
A Dual Approach to Scalable Verification of Deep Networks (2018, background): “Under a different set of assumptions, the existence of adversarial examples with high probability is also established in Fawzi et al. [2018]. In Wang et al.”
Are adversarial examples inevitable? (2018, background): “Fawzi et al. (2018) study adversarial susceptibility of datasets under the assumption that they are produced by a generative model that maps random Gaussian vectors onto images.”
Adversarial Examples Are a Natural Consequence of Test Error in Noise (2019, methodology): “This observation was also made in Fawzi et al. (2016; 2018b).”
The Curse of Concentration in Robust Learning: Evasion and Poisoning Attacks from Concentration of Measure (2018, methodology): “This above argument, is indeed inspired by the intuition behind the previous results of (Gilmer et al. 2018; Fawzi, Fawzi, and Fawzi 2018), and (Diochnos, Mahloujifar, and Mahmoody 2018) all of which use isoperimetric inequalities for specific metric probability spaces to prove limitations of…”
A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance (2019, background): “(2018) which shows that any measurable predictor on the unit hypercube [0, 1] will have an adversarial example if perturbed strongly enough, and Fawzi et al. (2018) which shows that any predictor on a data distribution supported on a hypersphere is vulnerable to small L2 noise. While Fawzi et al. (2”
Rademacher Complexity for Adversarially Robust Generalization (2018, background): “[20] construct examples of distributions with large latent variable space such that adversarially robust classifiers do not exist; here we argue that these examples may not explain the fact that adversarially perturbed images can usually be recognized by humans.”
A Closer Look at Accuracy vs. Robustness (2020, background): “In other words, we have further evidence that well-curated benchmark datasets are insufﬁcient to demonstrate a tradeoff between robustness and accuracy, in contrast to predictions of an inevitable tradeoff [11, 12, 16, 57].”
Robustness via Curvature Regularization, and Vice Versa (2018, background): “On the analysis front, many works have analyzed the existence of adversarial examples, and proposed several hypotheses for their existence [6, 8, 22, 5, 12].”
Computational Limitations in Robust Classification and Win-Win Results (2019, background): “A line of work which (Gilmer et al., 2018; Fawzi et al., 2018; Mahloujifar and Mahmoody, 2018) shows that for certain learning tasks and distributions (eg spheres in Rn or product distributions), due to concentration of measure adversarial examples exist close to points in the distribution and can a”