Benchmarking Adversarial Robustness on Image Classification
By Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Zihao Xiao, Jun Zhu. Computer Vision and Pattern Recognition, 2020
304 citations recorded by the source corpus.
Deep neural networks are vulnerable to adversarial examples, which becomes one of the most important research problems in the development of deep learning. While a lot of efforts have been made in recent years, it is of great significance to perform correct and complete evaluations of the adversarial attack and defense algorithms. In this paper, we establish a comprehensive, rigorous, and coherent benchmark to evaluate adversarial robustness on image classification tasks. After briefly reviewing plenty of representative attack and defense methods, we perform large-scale experiments with two robustness curves as the fair-minded evaluation criteria to fully understand the performance of these methods. Based on the evaluation results, we draw several important findings that can provide insights for future research, including: 1) The relative robustness between models can change across different attack configurations, thus it is encouraged to adopt the robustness curves to evaluate adversarial robustness; 2) As one of the most effective defense techniques, adversarial training can generalize across different threat models; 3) Randomization-based defenses are more robust to query-based black-box attacks.
View Paper: https://www.semanticscholar.org/paper/25e9fa483a048607131a5a0e3287e8f457fb4807
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
- Matthias Hein
- Francesco Croce
- Maksym Andriushchenko
- Shuicheng Yan
- Zekai Wang
- Tianyu Pang
- Vikash Sehwag
- Sylvestre-Alvise Rebuffi
Citations in the corpus
- Better Diffusion Models Further Improve Adversarial Training (2023, background): “Dong et al. (2020) perform large-scale experiments to generate robustness curves for evaluating typical defenses; Tang et al. (2021) provide comprehensive studies on how architecture design and training techniques affect robustness.”
- RobustBench: a standardized adversarial robustness benchmark (2020, background): “[33] have provided an evaluation of a few defenses (in particular, 3 for `∞116 and 2 for `2-norm on CIFAR-10) against multiple commonly used attacks.”