Fixing Data Augmentation to Improve Adversarial Robustness
By Sylvestre-Alvise Rebuffi, Sven Gowal, Dan A. Calian, Florian Stimberg, Olivia Wiles, Timothy A. Mann. arXiv.org, 2021
319 citations recorded by the source corpus.
Adversarial training suffers from robust overfitting, a phenomenon where the robust test accuracy starts to decrease during training. In this paper, we focus on both heuristics-driven and data-driven augmentations as a means to reduce robust overfitting. First, we demonstrate that, contrary to previous findings, when combined with model weight averaging, data augmentation can significantly boost robust accuracy. Second, we explore how state-of-the-art generative models can be leveraged to artificially increase the size of the training set and further improve adversarial robustness. Finally, we evaluate our approach on CIFAR-10 against $\ell_\infty$ and $\ell_2$ norm-bounded perturbations of size $\epsilon = 8/255$ and $\epsilon = 128/255$, respectively. We show large absolute improvements of +7.06% and +5.88% in robust accuracy compared to previous state-of-the-art methods. In particular, against $\ell_\infty$ norm-bounded perturbations of size $\epsilon = 8/255$, our model reaches 64.20% robust accuracy without using any external data, beating most prior works that use external data.
Subfields: Artificial Intelligence, Signal Processing
View Paper: https://www.semanticscholar.org/paper/762752eb9a9a92b028026b17c46d50474ddf3f06
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Matthias Hein
- Jacob Steinhardt
- Maksym Andriushchenko
- Francesco Croce
- Tianyu Pang
- Prateek Mittal
- Bernt Schiele
- Percy Liang
In Signal Processing
- Matthias Hein
- Maksym Andriushchenko
- Francesco Croce
- Jacob Steinhardt
- Dan Hendrycks
- Tianyu Pang
- Prateek Mittal
- Chaowei Xiao
Citations in the corpus
- Better Diffusion Models Further Improve Adversarial Training (2023, methodology): “Note that unlike Rebuffi et al. (2021) and Gowal et al. (2021) that use unconditional DDPM, the pseudo-labels of the generated images are directly determined by the class conditioning in our implementation.”
- RobustBench: a standardized adversarial robustness benchmark (2020, methodology): “[111], but there are also defenses which pursue additional goals alongside adversarial robustness at the fixed threshold we use: e.”
- Diffusion Models for Adversarial Purification (2022, methodology): “(2018), adversarial training has become one of the most successful defense methods in neural networks against adversarial attacks (Gowal et al., 2020; Rebuffi et al., 2021).”
- Unsolved Problems in ML Safety (2021, background): “[149] Sylvestre-Alvise Rebuffi, Sven Gowal, D.”