Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks

By Francesco Croce, Matthias Hein. 2020

2,457 citations recorded by the source corpus.

The field of defense strategies against adversarial attacks has significantly grown over the last years, but progress is hampered as the evaluation of adversarial defenses is often insufficient and thus gives a wrong impression of robustness. Many promising defenses could be broken later on, making it difficult to identify the state-of-the-art. Frequent pitfalls in the evaluation are improper tuning of hyperparameters of the attacks, gradient obfuscation or masking. In this paper we first propose two extensions of the PGD-attack overcoming failures due to suboptimal step size and problems of the objective function. We then combine our novel attacks with two complementary existing ones to form a parameter-free, computationally affordable and user-independent ensemble of attacks to test adversarial robustness. We apply our ensemble to over 50 models from papers published at recent top machine learning and computer vision venues. In all except one of the cases we achieve lower robust test accuracy than reported in these papers, often by more than $10\%$, identifying several broken defenses.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Molecular Biology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Bernt Schiele
2. Jun Zhu
3. Alan L. Yuille
4. Matthias Hein
5. Quoc V. Le
6. Yoshua Bengio
7. Percy Liang
8. Hang Su

In Electrical and Electronic Engineering

1. Matthias Hein
2. Tianyu Pang
3. Jun Zhu
4. Sven Gowal
5. Timothy A. Mann
6. Bernt Schiele
7. David Stutz
8. Hang Su

In Molecular Biology

1. Jun Zhu
2. Sven Gowal
3. Timothy A. Mann
4. Tianyu Pang
5. Hang Su
6. Xiao Yang
7. Yinpeng Dong
8. Maksym Andriushchenko

In Safety, Robustness & Interpretability

1. Matthias Hein
2. Tianyu Pang
3. Jun Zhu
4. Yinpeng Dong
5. Hang Su
6. Sven Gowal
7. Timothy A. Mann
8. David Stutz

Citations in the corpus

• Adversarial Example Games (2020, methodology): “For example, in the popular whitebox threat model, the attacker is assumed to have full access to the model f ’s parameters and outputs [62, 30, 51, 53, 11, 16, 17].”
• Do Wider Neural Networks Really Help Adversarial Robustness? (2020, methodology): “In particular, we choose to evaluate our best models on the AutoAttack algorithm [17], which is an ensemble attack method that contains four different white-box and black-box attacks for the best attack performances.”
• Boosting Adversarial Training with Hypersphere Embedding (2020, methodology): “We evaluate our enhanced models under two stronger benchmark attacks including RayS [11] and AutoAttack [15] on CIFAR-10.”
• On the Robustness of Vision Transformers to Adversarial Examples (2021, methodology): “Overall, CNNs have an expansive body of literature related to adversarial attacks [6, 11, 10] and defenses [23, 5, 34].”
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples (2020, methodology): “With additional unlabeled data, we use a batch size of 1024 with 512 samples from C IFAR -10 and 512 samples from a subset of 500K images extracted from the tiny images dataset 80M-T I (Torralba et al., 2008) (Croce & Hein, 2020) and M ULTI T ARGETED (Gowal et al., 2019).”
• Better Diffusion Models Further Improve Adversarial Training (2023, methodology): “…the need for external datasets or additional training time per epoch, our adversarially trained WRN-70-16 models (Zagoruyko & Komodakis, 2016) achieve new SOTA robust accuracy on CIFAR-10/CIFAR-100 under AutoAttack (AA) (Croce & Hein, 2020), associated with a large improvement in clean accuracy.”
• RobustBench: a standardized adversarial robustness benchmark (2020, methodology): “Moreover, 268 we measure robust accuracy, in the same threat model used on CIFAR-10, using AutoAttack [26] 269 (see Fig.”
• Square Attack: a query-efficient black-box adversarial attack via random search (2019, methodology): “Additionally, the subsequent work of [21] uses the Square Attack as part of their AutoAttack where they show that the Square Attack outperforms other whitebox attacks on 9 out of 9 MNIST models they evaluated.”
• Diffusion Models for Adversarial Purification (2022, methodology): “We first compare DiffPure with the state-of-the-art adversarial training methods reported by RobustBench (Croce et al., 2020), against the ∞ and (cid:96) 2 threat models, respectively.”
• Relating Adversarially Robust Generalization to Flat Minima (2021, methodology): “AutoAttack [11]) but does not result in visually flatter minima.”
• Data Augmentation Can Improve Robustness (2021, methodology): “Finally, we report the robust test accuracy against a mixture of A UTO A TTACK [10] and M ULTI T ARGETED [19], which is denoted by AA+MT.”
• Fixing Data Augmentation to Improve Adversarial Robustness (2021, methodology): “…step on the cross-entropy loss (Croce & Hein, 2020b), • AUTOPGD-T, a targeted attack using PGD with an adaptive step on the difference of logits ratio (Croce & Hein, 2020b), • FAB-T, a targeted attack which minimizes the norm of adversarial perturbations (Croce & Hein, 2020a), • SQUARE, a…”
• Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack (2019, methodology): “Finally, (Croce & Hein, 2020) use FAB-attack together with other white- and black-box attacks to evaluate the robustness of over 50 classifiers trained with recently proposed adversarial defenses wrt l∞ and l2 on different datasets.”
• Bag of Tricks for Adversarial Training (2020, methodology): “Croce & Hein (2020b) propose AutoAttack, which is an ensemble of four selected attacks.”
• Cross-Entropy Loss Functions: Theoretical Analysis and Applications (2023, methodology): “We report the clean accuracy, that is the standard classification accuracy on the test set, and the robust accuracy with `∞-norm perturbations bounded by γ = 8/255 under PGD attack, measured by PGD40margin on the full test set, as well as under AutoAttack (Croce & Hein, 2020) (https://github.com/ fr”
• Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models (2020, background): “Our evaluation of the JEM model shows significantly less defense that the original evaluation, and our results are consistent with the JEM evaluation in [7] that uses the same attack methodology as we do.”
• Geometry-aware Instance-reweighted Adversarial Training (2020, methodology): “The evaluations are on the full set of the AA in (Croce & Hein, 2020) with test = 0.031, which keeps the same as training.”