Adversarial Training Can Hurt Generalization

By Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C. Duchi, Percy Liang. 2019

257 citations recorded by the source corpus.

While adversarial training can improve robust accuracy (against an adversary), it sometimes hurts standard accuracy (when there is no adversary). Previous work has studied this tradeoff between standard and robust accuracy, but only in the setting where no predictor performs well on both objectives in the infinite data limit. In this paper, we show that even when the optimal predictor with infinite data performs well on both objectives, a tradeoff can still manifest itself with finite data. Furthermore, since our construction is based on a convex learning problem, we rule out optimization concerns, thus laying bare a fundamental tension between robustness and generalization. Finally, we show that robust self-training mostly eliminates this tradeoff by leveraging unlabeled data.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Adversarial Training for Large Neural Language Models (2020, background): “While adversarial training has been successful in mitigating adversarial attacks, past work often encounters an apparent conﬂict between generalization and robustness (Raghunathan et al., 2019, 2020; Min et al., 2020), as adversarial training could hurt generalization performance.”
Feature Purification: How Adversarial Training Performs Robust Deep Learning (2020, background): “Other theories about adversarial examples focus on how adversarial training might require more training data comparing to clean training [84], and might decrease clean training accuracy [80].”
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty (2019, background): “However, this tends to increase training time by an order of magnitude and substantially degrades accuracy on non-adversarial images (Raghunathan et al., 2019).”
Adversarial Examples Improve Image Recognition (2019, background): “Under supervised learning settings, adversarial training is typically considered hurting accuracy on clean images [32], e.”
A Closer Look at Accuracy vs. Robustness (2020, methodology): “While a number of defenses have been proposed [10, 32, 46, 65], they are known to hurt test accuracy on many datasets [41, 32, 68].”
Recent Advances in Adversarial Training for Adversarial Robustness (2021, unknown): “Experiments in [Raghunathan et al., 2019] support this statement, where additional unlabeled data is proved to help mitigate the tradeoff.”
Trustworthy AI: From Principles to Practices (2021, background): “Recent research [271, 331], however, has noted that improving robustness by adversarial training may reduce the testing accuracy and leads to worse generalization.”
Improved OOD Generalization via Adversarial Training and Pre-training (2021, background): “However, an increasing perturbation size r improves the input-robustness, but harms the clean accuracy (Raghunathan et al., 2019).”
Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin (2019, background): “A number of recent works provide negative results for adversarially robust generalization [Tsipras et al., 2018, Montasser et al., 2019, Yin et al., 2018, Raghunathan et al., 2019].”
Property-driven Training: All You (N)Ever Wanted to Know About (2021, background): “2019), which is frequently referred to as the accuracy-robustness tradeoff for neural networks (although this effect has been observed to disappear as the size of the training dataset grows (Raghunathan et al. 2019)).”
Do Wider Neural Networks Really Help Adversarial Robustness? (2020, background): “Previous works [70, 61, 52] have argued that there exists a trade-off between natural accuracy and robust accuracy.”
Relating Adversarially Robust Generalization to Flat Minima (2021, background): “However, AT is slow [71] and suffers from increased sample complexity [49] as well as reduced (clean) accuracy [59, 53, 72, 45].”
Precise Tradeoffs in Adversarial Training for Linear Regression (2020, background): “A few recent papers have begun to answer some of these questions in speciﬁc settings [TSE + 18, ZYJ + 19, RXY + 19].”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “This is different from adversarial training, which generally results in sacrificing model accuracy on the clean images to gain robustness to adversarial examples [29], [281].”
Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness (2019, background): “Although recent works show a fundamental trade-off between robust and standard accuracy in constructed �p perturbation settings [41, 48, 36], we additionally show that this is fundamentally different for spatial transformations due to their group structure.”
The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization (2020, background): “Raghunathan et al. [2019] showed empirical evidence that adversarial training could hurt the standard accuracy, despite its improvement on the robust accuracy.”
The Origins and Prevalence of Texture Bias in Convolutional Neural Networks (2019, background): “As with adversarial training [93, 73], augmentation strategies that improve robustness can degrade clean accuracy [65], and tradeoffs exist between robustness to different corruptions [102].”