Overfitting in adversarially robust deep learning

By Eric Wong, Leslie Rice, Zico Kolter. International Conference on Machine Learning, 2020

962 citations recorded by the source corpus.

It is common practice in deep learning to use overparameterized networks and train for as long as possible; there are numerous studies that show, both theoretically and empirically, that such practices surprisingly do not unduly harm the generalization performance of the classifier. In this paper, we empirically study this phenomenon in the setting of adversarially trained deep networks, which are trained to minimize the loss under worst-case adversarial perturbations. We find that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training across multiple datasets (SVHN, CIFAR-10, CIFAR-100, and ImageNet) and perturbation models ($\ell_\infty$ and $\ell_2$). Based upon this observed effect, we show that the performance gains of virtually all recent algorithmic improvements upon adversarial training can be matched by simply using early stopping. We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting. Finally, we study several classical and modern deep learning remedies for overfitting, including regularization and data augmentation, and find that no approach in isolation improves significantly upon the gains achieved by early stopping. All code for reproducing the experiments as well as pretrained model weights and training logs can be found at this https URL.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/2eda2921a8da4b325f9d05f556594a5884c398a7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Boosting Adversarial Training with Hypersphere Embedding (2020, background): “This causes the training process to devote noneffective efforts to increasing ‖z‖ for easy examples and consequently overlook the hard ones, which leads to vicious circles and could degrade the model robustness against strong adversarial attacks [51, 59].”
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples (2020, methodology): “For consistency with prior work on adversarial robustness (Madry et al., 2017; Rice et al., 2020; Zhang et al., 2019a; Uesato et al., 2019), we use Wide ResNets (W RN s) (He et al., 2015; Zagoruyko & Komodakis, 2016).”
Data Augmentation Can Improve Robustness (2021, methodology): “[44], they also make the conclusion that data augmentations beyond random padding-and-cropping do not improve robustness.”
Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks (2020, neutral): “13 4 (Rice et al., 2020) 85.”
Geometry-aware Instance-reweighted Adversarial Training (2020, methodology): “1 shows that GAIRAT can relieve the undesirable robust overfitting (Rice et al., 2020) of the minimax-based adversarial training (Madry et al.”
Bag of Tricks for Adversarial Training (2020, methodology): “Besides, the reported robustness of PGD-AT in Rice et al. (2020) is much higher than it in Madry et al. (2018), even without early-stopping.”
Fixing Data Augmentation to Improve Adversarial Robustness (2021, methodology): “• To the contrary of Rice et al. (2020); Wu et al. (2020); Gowal et al. (2020) which all tried data augmentation techniques without success, we are able to use any of these three aforementioned techniques to obtain new state-of-the-art robust accuracies.”
Improving Robustness using Generated Data (2021, methodology): “…two) models for each hyperparameter setting, perform early stopping for each model on a separate validation set of 1024 samples using P GD 40 (i.e., PGD with 40 gradient ascent steps) similarly to Rice et al. [61] and pick the best model by evaluating the robust accuracy on the same validation set.”
Cross-Entropy Loss Functions: Theoretical Analysis and Applications (2023, methodology): “We used early stopping on a held-out validation set of 1,024 samples by evaluating its robust accuracy throughout training with 40-step PGD on the margin loss, denoted by PGD(40) margin, and selecting the best check-point (Rice et al., 2020).”
Attacks Which Do Not Kill Training Make Adversarial Learning Stronger (2020, methodology): “…training (Cai et al., 2018; Wang et al., 2019), computationally efficient adversarial learning (Shafahi et al., 2019; Zhang et al., 2019a; Wong et al., 2020), ensemble adversarial training (Tramr et al., 2018; Pang et al., 2019), and adversarial training by utilizing unlabeled data…”
Deep Repulsive Prototypes for Adversarial Robustness (2021, methodology): “Therefore, we also report on a new benchmark, Early Stop-R, which is obtained using the model parameters shared in the project’s repository by Rice, Wong, and Kolter.”
Better Diffusion Models Further Improve Adversarial Training (2023, methodology): “For AT, however, Rice et al. (2020) reveal the phenomenon of robust overfitting: robust accuracy degrades rapidly on the test set while it continues to increase on the training set.”
Recent Advances in Adversarial Training for Adversarial Robustness (2021, background): “Other than CIFAR-10, similar experimental results are observed on multiple datasets, such as SVHN, CIFAR-100, and ImageNet [Rice et al. , 2020].”
Are We Learning Yet? A Meta Review of Evaluation Failures Across Machine Learning (2021, background): “• And in adversarial robustness, where early-stopping with standard projected gradient descent was found to give performance on par with newer alternatives [127].”
Adversarial Weight Perturbation Helps Robust Generalization (2020, methodology): “, flattening the input loss landscape) to obtain a certain of robustness, but its robustness is still far from satisfactory because of the huge robust generalization gap [39, 37], for example, an adversarially trained PreAct ResNet-18 [14] on CIFAR-10 [20] only has 43% test robustness, even it has a”
RobustBench: a standardized adversarial robustness benchmark (2020, background): “Thus one can expect lower privacy leakage in robust networks as 330 multiple previous works have explicitly aimed to reduce the generalization gap in robust training using 331 techniques such as early stopping [102, 152, 46].”
A Closer Look at Accuracy vs. Robustness (2020, methodology): “The fact that AT may sometimes overﬁt has been previously observed by [41, 43, 55]; in particular, Rice et al. [43] also experiments with a few generalization methods (but not dropout) and observes that only early stopping helps overcome overﬁtting to a certain degree.”