Robustness May Be at Odds with Accuracy

By Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, Aleksander Madry. 2018

1,971 citations recorded by the source corpus.

We show that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization. Specifically, training robust models may not only be more resource-consuming, but also lead to a reduction of standard accuracy. We demonstrate that this trade-off between the standard accuracy of a model and its robustness to adversarial perturbations provably exists in a fairly simple and natural setting. These findings also corroborate a similar phenomenon observed empirically in more complex settings. Further, we argue that this phenomenon is a consequence of robust classifiers learning fundamentally different feature representations than standard classifiers. These differences, in particular, seem to result in unexpected benefits: the representations learned by robust models tend to align better with salient data characteristics and human perception.

Subfields: Artificial Intelligence, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety, Robustness & Interpretability

Citations in the corpus

Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations (2020, methodology): “We formalize these observations using a simple classification task inspired by (Tsipras et al., 2019).”
Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, neutral): “[59] and Zhang et al.”
Adversarial Training and Robustness for Multiple Perturbations (2019, methodology): “To illustrate, we recall two classifiers from [41] that operate on disjoint feature sets.”
Adversarial Training for Free! (2019, background): “We see some dropoff in accuracy for small values of m. Note that a small compromise in accuracy is acceptable given a large increase in robustness due to the fundamental tradeoffs between robustness and generalization [Tsipras et al., 2018, Zhang et al., 2019a, Shafahi et al., 2019b].”
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness (2019, background): “Hence, we argue that the often-discussed tradeoff between accuracy and robustness (see Tsipras et al. (2019) for a recent treatment) should in fact take into account at least three notions: accuracy, sensitivity, and invariance.”
Certified Adversarial Robustness via Randomized Smoothing (2019, background): “This observation echoes the finding in Tsipras et al. (2019) that adversarially trained networks with higher robust accuracy tend to have lower”
Generative Counterfactual Introspection for Explainable Deep Learning (2019, background): “One possible explanation for such an observation is that the classifier learns these patterns from the training data (the model may try to obtain better accuracy by picking up features that are not aligned with human perception [32]).”
Explainable machine learning in deployment (2019, neutral): “with respect to the input shows that the model is focusing on relevant portions of the images, confirming recent research [21, 30, 64].”
Excessive Invariance Causes Adversarial Vulnerability (2018, background): “The difference to human perception revealed by adversarial examples fundamentally questions which statistics the network use to base its decisions (Jo & Bengio, 2017; Tsipras et al., 2018).”
Theoretically Principled Trade-off between Robustness and Accuracy (2019, background): “A recent work demonstrates the existence of trade-off between accuracy and robustness (Tsipras et al., 2019).”
A Fourier Perspective on Model Robustness in Computer Vision (2019, background): “Perhaps the reason adversarially trained models have semantically meaningful gradients (Tsipras et al., 2018) Figure 7 can drop accuracies to less than 30%.”
Unlabeled Data Improves Adversarial Robustness (2019, neutral): “[47], Fawzi et al.”
Identifying Statistical Bias in Dataset Replication (2020, background): “Previous work has characterized cross-dataset generalization(Torralba & Efros, 2011), and explored the impact of synthetic perturbations on generalization, such as adversarial examples (Kurakin et al., 2016; Tsipras et al., 2019; Ilyas et al., 2019; Su et al., 2018) or various other corruption robus”