Disentangling Adversarial Robustness and Generalization

By David Stutz, Matthias Hein, Bernt Schiele. 2018

329 citations recorded by the source corpus.

Obtaining deep networks that are robust against adversarial examples and generalize well is an open problem. A recent hypothesis even states that both robust and accurate models are impossible, i.e., adversarial robustness and generalization are conflicting goals. In an effort to clarify the relationship between robustness and generalization, we assume an underlying, low-dimensional data manifold and show that: 1. regular adversarial examples leave the manifold; 2. adversarial examples constrained to the manifold, i.e., on-manifold adversarial examples, exist; 3. on-manifold adversarial examples are generalization errors, and on-manifold adversarial training boosts generalization; 4. regular robustness and generalization are not necessarily contradicting goals. These assumptions imply that both robust and accurate models are possible. However, different models (architectures, training strategies etc.) can exhibit different robustness and generalization characteristics. To confirm our claims, we present extensive experiments on synthetic data (with known manifold) as well as on EMNIST, Fashion-MNIST and CelebA.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial Training for Large Neural Language Models (2020, background): “Additionally, we hypothesize that by perturbing the embedding space rather than the input space, adversarial training in NLP might in-advertently bias toward on-manifold perturbation than regular perturbation, which helps generalization (Stutz et al., 2019).”
Adversarial Examples Are Not Bugs, They Are Features (2019, background): “From this point of view, it is natural to treat adversarial robustness as a goal that can be disentangled and pursued independently from maximizing accuracy [Mad+18; SHS19; Sug+19], either through improved standard regularization methods [TG16] or pre/post-processing of network inputs/outputs [Ues+1”
Feature Purification: How Adversarial Training Performs Robust Deep Learning (2020, background): “In particular, we found that the following fundamental questions remain largely unaddressed: Questions Why do adversarial examples exist when we train the neural networks using the original training data set? How can adversarial training further “robustify” the trained neural networks against these ”
Why ReLU Networks Yield High-Confidence Predictions Far Away From the Training Data and How to Mitigate the Problem (2018, background): “eresting as one can hope that the reduction of the conﬁdence for out-of-distribution images also reduces the conﬁdence of adversarial samples as typically adversarial samples are oﬀ the data manifold [33] and thus are also out-of-distribution sam5 Noise Samples MNIST Adversarial Noise MNIST for Plai”
Adversarial Examples Improve Image Recognition (2019, background): “Other works try to explain this tradeoff phenomenon from the perspective of the increased sample complexity of adversary [37, 25, 28], the limited amount of training data [1, 27, 34, 44, 48], or network overparameterization [31].”
Learning to Learn Single Domain Generalization (2020, background): “[47] pointed out that on-manifold adversarial training boosts generalization, and hence models with both robustness and generalization can be obtained at the same time.”
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review (2019, background): “Moreover, a number of studies reported the relation between these two properties (Tsipras et al., 2018; Su et al., 2018; Stutz et al., 2019; Zhang et al., 2019b).”
Better Diffusion Models Further Improve Adversarial Training (2023, background): “It is acknowledged that the availability of more data is critical to the performance of adversarially trained models (Schmidt et al., 2018; Stutz et al., 2019).”
Relating Adversarially Robust Generalization to Flat Minima (2021, background): “However, AT is slow [71] and suffers from increased sample complexity [49] as well as reduced (clean) accuracy [59, 53, 72, 45].”
The Intrinsic Dimension of Images and Its Impact on Learning (2021, methodology): “In this section, we describe a ﬁnal technique for studying the effect of intrinsic dimensionality on sample complexity on the recently proposed FONTS dataset (Stutz et al., 2019).”
Bridging Adversarial Robustness and Gradient Interpretability (2019, methodology): “Following previous works Tsipras et al. (2018) and Stutz et al. (2018), we first trained DNNs against adversarial attacks which maximize the training loss / cross entropy (XEnt) loss or the CW surrogate objective proposed by Carlini & Wagner (2017).”
Recent Advances in Adversarial Training for Adversarial Robustness (2021, background): “The shreds of evidence in [Yang et al. , 2020b; Stutz et al. , 2019] show that adversarial training might not be the optimal solution for obtaining models with desirable robustness and accuracy.”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “There is also a debate in the literature of associating robustness of neural models to their generalization [315].”