Adversarial Training and Robustness for Multiple Perturbations

By Florian Tramèr, Dan Boneh. 2019

426 citations recorded by the source corpus.

Defenses against adversarial examples, such as adversarial training, are typically tailored to a single perturbation type (e.g., small $\ell_\infty$-noise). For other perturbations, these defenses offer no guarantees and, at times, even increase the model's vulnerability. Our aim is to understand the reasons underlying this robustness trade-off, and to train models that are simultaneously robust to multiple perturbation types. We prove that a trade-off in robustness to different types of $\ell_p$-bounded and spatial perturbations must exist in a natural and simple statistical setting. We corroborate our formal analysis by demonstrating similar robustness trade-offs on MNIST and CIFAR10. Building upon new multi-perturbation adversarial training schemes, and a novel efficient attack for finding $\ell_1$-bounded adversarial examples, we show that no model trained against multiple attacks achieves robustness competitive with that of models trained on each attack individually. In particular, we uncover a pernicious gradient-masking phenomenon on MNIST, which causes adversarial training with first-order $\ell_\infty, \ell_1$ and $\ell_2$ adversaries to achieve merely $50\%$ accuracy. Our results question the viability and computational scalability of extending adversarial robustness, and adversarial training, to multiple perturbation types.

Subfields: Artificial Intelligence, Molecular Biology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, background): “Encouragingly, the past few years have seen substantial progress in robustness to these distribution shifts, e.g., see [13, 28, 34, 55, 57, 66, 93, 96, 105, 114, 115] among many others.”
Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations (2020, background): “Increasing model robustness to `p-noise also leads to other tradeoffs, such as reduced accuracy (Tsipras et al., 2019) or reduced robustness to other small perturbations (Yin et al., 2019; Tramèr & Boneh, 2019; Kang et al., 2019).”
Square Attack: a query-efficient black-box adversarial attack via random search (2019, background): “It is known that the PGD is ineffective since it suffers from gradient masking [53].”
A Simple Way to Make Neural Networks Robust Against Diverse Image Corruptions (2020, background): “Tramèr et al. [40] also argue in favor of a trade-oﬀ between diﬀerent robustness types.”
Adversarial Robustness Against the Union of Multiple Perturbation Models (2019, background): “Finally, Tramèr and Boneh [2019] study the theoretical and empirical trade-offs of adversarial robustness in various settings when defending against multiple adversaries, however, they use a rotation and translation adversary instead of an (cid:96) 2 adversary for CIFAR10.”
Bag of Tricks for Adversarial Training (2020, background): “Evaluating robustness on these datasets can reﬂect the generality of the proposed defenses, and avoid overﬁtting to certain attacking patterns (Engstrom et al., 2019; Tramèr & Boneh, 2019).”
RobustBench: a standardized adversarial robustness benchmark (2020, background): “bounded by `0, `1-norm or adversarial patches [11, 26, 93, 29], (2) multiple `p-norm perturbations [140, 90], (3) adversarially optimized common corruptions [68, 69], (4) a broad set of perturbations unseen during training [81].”
On Adaptive Attacks to Adversarial Example Defenses (2020, background): “To make models robust to (cid:96) 0 -attacks, one option is to train a model against (cid:96) 1 -attacks (Tramèr & Boneh, 2019).”
The Origins and Prevalence of Texture Bias in Convolutional Neural Networks (2019, background): “However, models that are robust to adversarial examples generated with respect to the ℓ p norm are not necessarily robust to other forms of imperceptible perturbations [98, 92].”
A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inference (2020, methodology): “03 (Madry et al., 2017), || v || 1 ≤ 8 (Tramèr & Boneh, 2019) and || v || 2 ≤ 0 .”
Relating Adversarially Robust Generalization to Flat Minima (2021, neutral): “Originally proposed in different variants in [56, 40, 24], it received considerable attention in [37, 16] and has been extended in various ways: [33, 7, 2] utilize interpolated or unlabeled examples, [57, 38] achieve robustness against multiple threat models, [54, 32, 64] augment AT with a reject op”
Blind Backdoors in Deep Learning Models (2020, background): “Further, certified defenses are not robust against attacks that use a different metric than the defense [89] and can break a model [88] because some small changes—e.”
Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack (2019, methodology): “Both properties lead to the fact that it is robust to gradient masking which can be a problem for PGD (Tramèr & Boneh, 2019).”