Adversarially Robust Generalization Requires More Data

By Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, Aleksander Madry. 2018

867 citations recorded by the source corpus.

Machine learning models are often susceptible to adversarial perturbations of their inputs. Even small perturbations can cause state-of-the-art classifiers with high "standard" accuracy to produce an incorrect prediction with high confidence. To better understand this phenomenon, we study adversarially robust learning from the viewpoint of generalization. We show that already in a simple natural data model, the sample complexity of robust learning can be significantly larger than that of "standard" learning. This gap is information theoretic and holds irrespective of the training algorithm or the model family. We complement our theoretical results with experiments on popular image classification datasets and show that a similar gap exists here as well. We postulate that the difficulty of training robust classifiers stems, at least partially, from this inherently larger sample complexity.

Subfields: Artificial Intelligence, Molecular Biology, Scaling & Learning Theory, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Scaling & Learning Theory

In Safety, Robustness & Interpretability

Citations in the corpus

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (2019, background): “For some simple models, adversarial robustness requires an increase in the training set size that is polynomial in d (Schmidt et al., 2018).”
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification (2021, background): “On the other hand, recurring ﬁndings reveal that such models are brittle when provided with out-of-domain [1, 2, 3] or adversarial examples [4, 5, 6, 7, 8, 9, 10], and make silly mistakes that contradict basic notions of human common sense.”
Feature Purification: How Adversarial Training Performs Robust Deep Learning (2020, background): “Other theories about adversarial examples focus on how adversarial training might require more training data comparing to clean training [65], and might decrease clean”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, background): “based on both empirical and theoretical analysis on toy classification tasks. Moreover, the overfitting phenomenon can be worse for a robust model when evaluating its accuracy on adversarial examples [39, 46]. Thus, compared with the natural models, the robust models might leak more membership infor”
Adversarial Training and Robustness for Multiple Perturbations (2019, background): “Thus, multi-perturbation robustness increases the adversarial generalization gap [30].”
Certified Adversarial Robustness via Randomized Smoothing (2019, background): “Recent theoretical work has taken steps towards addressing this question (Mahloujifar et al., 2019; Gilmer et al., 2018b; Fawzi et al., 2018; Tsipras et al., 2019; Schmidt et al., 2018; Shafahi et al., 2019; Dohma- tob, 2018; Bubeck et al., 2018b).”
Towards the first adversarially robust neural network model on MNIST (2018, background): “can also be achieved with a simple input quantisation because of the binary nature of single pixels in MNIST (which are typically either completely black or white) (Schmidt et al., 2018).”
Using Pre-Training Can Improve Model Robustness and Uncertainty (2019, background): “In a later work, Schmidt et al. (2018) argue further progress on this problem may require signiﬁcantly more task-speciﬁc data.”
Pre-Trained Models: Past, Present and Future (2021, background): “In previous work, it was shown that a higher sample complexity is needed in order to achieve adversarial robustness for neural networks (Schmidt et al., 2018).”
Excessive Invariance Causes Adversarial Vulnerability (2018, background): “The reasons for these failures and how they may be avoided or at least mitigated is a prime research area (Schmidt et al., 2018; Gilmer et al., 2018b; Bubeck et al., 2018).”
Robustness May Be at Odds with Accuracy (2018, background): “Schmidt et al. (2018) study the generalization aspect of adversarially robustness.”
Theoretically Principled Trade-off between Robustness and Accuracy (2019, background): “Despite a large literature on the analysis of robust error in terms of generalization (Schmidt et al., 2018; Cullina et al., 2018; Yin et al., 2018) and computational complexity (Bubeck et al.”
Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty (2019, methodology): “Importantly, these methods can improve robustness and uncertainty estimation without requiring larger models or additional data [Schmidt et al., 2018, Kurakin et al., 2017].”
Unlabeled Data Improves Adversarial Robustness (2019, methodology): “[41] show a sample complexity barrier to robustness in a stylized setting.”