Theoretically Principled Trade-off between Robustness and Accuracy
By Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, Michael I. Jordan. 2019
3,092 citations recorded by the source corpus.
We identify a trade-off between robustness and accuracy that serves as a guiding principle in the design of defenses against adversarial examples. Although this problem has been widely studied empirically, much remains unknown concerning the theory underlying this trade-off. In this work, we decompose the prediction error for adversarial examples (robust error) as the sum of the natural (classification) error and boundary error, and provide a differentiable upper bound using the theory of classification-calibrated loss, which is shown to be the tightest possible upper bound uniform over all probability distributions and measurable predictors. Inspired by our theoretical analysis, we also design a new defense method, TRADES, to trade adversarial robustness off against accuracy. Our proposed algorithm performs well experimentally in real-world datasets. The methodology is the foundation of our entry to the NeurIPS 2018 Adversarial Vision Challenge in which we won the 1st place out of ~2,000 submissions, surpassing the runner-up approach by $11.41\%$ in terms of mean $\ell_2$ perturbation distance.
Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Hardware and Architecture, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Percy Liang
- Mohit Bansal
- Jianfeng Gao
- Yoshua Bengio
- Quoc V. Le
- Sergey Levine
- Ruslan Salakhutdinov
- Dacheng Tao
In Electrical and Electronic Engineering
- Matthias Hein
- Tianyu Pang
- Percy Liang
- Maksym Andriushchenko
- Sven Gowal
- Francesco Croce
- Timothy A. Mann
- Jun Zhu
In Hardware and Architecture
- Matthias Hein
- Huan Zhang
- Cho-Jui Hsieh
- Tom Goldstein
- Francesco Croce
- Bernt Schiele
- Nicholas Carlini
- David Stutz
In Safety, Robustness & Interpretability
- Matthias Hein
- Nicholas Carlini
- Tianyu Pang
- Cho-Jui Hsieh
- Huan Zhang
- Francesco Croce
- Jun Zhu
- Percy Liang
Citations in the corpus
- Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, background): “Encouragingly, the past few years have seen substantial progress in robustness to these distribution shifts, e.g., see [13, 28, 34, 55, 57, 66, 93, 96, 105, 114, 115] among many others.”
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning (2020, methodology): “, trained models are able to resist adversarial attacks on clean images at the expense of performance [42, 73, 84].”
- Feature Purification: How Adversarial Training Performs Robust Deep Learning (2020, background): “This approach was reported in [13] as the only approach that can defend against carefully designed adversarial attacks, and many follow-up works are built upon it [14, 15].”
- Privacy Risks of Securing Machine Learning Models against Adversarial Examples (2019, methodology): “We consider the l ∞ -ball adversarial constraint throughout the paper, as it is widely adopted by most adversarial defense methods [16, 33, 34, 40, 50, 61, 66].”
- Adversarial Training for Free! (2019, background): “We see some dropoff in accuracy for small values of m. Note that a small compromise in accuracy is acceptable given a large increase in robustness due to the fundamental tradeoffs between robustness and generalization [Tsipras et al., 2018, Zhang et al., 2019, Shafahi et al., 2019].”
- The Ideal Continual Learner: An Agent That Never Forgets (2023, background): “, robustness (Zhang et al., 2019; Robey et al., 2021), safety (Paternain et al.”
- On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, positive): “…constant of a model – intuitively, a bound on how much the model’s response can change in proportion to a change in its input [5] – has proven to strengthen the decision surface around a point [22, 45, 46, 80, 91], preventing attacks of a given magnitude from changing the output of the classifier.”
- Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty (2019, methodology): “We also observe that auxiliary self-supervision can provide gains when complemented with stronger defenses like TRADES [Zhang et al., 2019] and is not broken by gradient-free attacks such as SPSA [Uesato et al., 2018].”
- Adversarial Examples Improve Image Recognition (2019, background): “Since the first discovery of the vulnerability of ConvNets to adversarial attacks [40], many efforts [2, 7, 15, 16, 18, 23, 29, 36, 42, 45, 50] have been made to improve network robustness.”
- Unlabeled Data Improves Adversarial Robustness (2019, methodology): “We use TRADES [56] for heuristic `1- robustness, and stability training [57] combined with randomized smoothing [9] for certified `2- robustness.”
- Are Labels Required for Improving Adversarial Robustness? (2019, methodology): “The TRADES approach in [43] improves these results by instead minimizing a surrogate loss which upper bounds the adversarial risk.”
- Causally motivated shortcut removal using auxiliary labels (2021, background): “Thus our work contributes to ongoing investigations about whether there is a trade-off between fari-ness and accuracy [Zhang et al., 2019, Calders et al., 2009, Johansson et al., 2019, Dutta et al., 2020, Zhao and Gordon, 2019].”
- Benchmarking Adversarial Robustness on Image Classification (2020, methodology): “Based on the results, we observe that under white-box attacks, the adversarially trained models (i.e., PGD-AT, TRADES) are more robust than other models, because they are trained on the worst-case adversarial examples.”
- Certified Adversarial Robustness with Additive Noise (2018, background): “After comparison, we realize the adversarial examples generated from Madry’s model [26] result in the strongest black-box attacks for both TRADES and STN.”