On Adaptive Attacks to Adversarial Example Defenses

By Florian Tramèr, Nicholas Carlini, Wieland Brendel, Aleksander Madry. Neural Information Processing Systems, 2020

979 citations recorded by the source corpus.

Adaptive attacks have (rightfully) become the de facto standard for evaluating defenses to adversarial examples. We find, however, that typical adaptive evaluations are incomplete. We demonstrate that thirteen defenses recently published at ICLR, ICML and NeurIPS---and chosen for illustrative and pedagogical purposes---can be circumvented despite attempting to perform evaluations using adaptive attacks. While prior evaluation papers focused mainly on the end result---showing that a defense was ineffective---this paper focuses on laying out the methodology and the approach necessary to perform an adaptive attack. We hope that these analyses will serve as guidance on how to properly perform adaptive attacks against defenses to adversarial examples, and thus will allow the community to make further progress in building more robust models.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/58c143069444c7dff4be53531a47efefc40be497

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Diffusion Models for Adversarial Purification (2022, methodology): “To this end, we use the BPDA+EOT attack (Tramer et al., 2020; Hill et al., 2021), an adaptive attack designed speciﬁ-cally for puriﬁcation methods (with stochasticity), to evaluate our method and baselines for a fair comparison.”
RobustBench: a standardized adversarial robustness benchmark (2020, methodology): “To prevent overadaptation of new defenses to AutoAttack, we welcome external evaluations based on adaptive attacks [142], especially where AutoAttack flags a potential overestimation of robustness.”
LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples (2023, background): “Therefore, we may avoid illegal adversarial attack from another view that we do not explicitly eliminate them, which is also impossible (Ilyas et al., 2019; Tramer et al., 2020), we may try to implicitly hide them and make the attack more hard (Xiao et al.”
JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (2024, background): “Finally, we note that proper evaluation of test-time defenses should rely on adaptive attacks , i.e., attacks tailored to the specific defense under evaluation (Tram ` er et al., 2020).”
Unsolved Problems in ML Safety (2021, background): “While defending against adversaries might seem to be a straightfoward problem, defenses are currently struggling to keep pace with attacks [8, 188], and much research is needed to discover how to ﬁx these longstanding weaknesses.”
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks (2023, positive): “To answer this question, we first introduce some notation to formalize the GCG attack.”
Deep Repulsive Prototypes for Adversarial Robustness (2021, background): “Unfortunately, none of these defences proved effective (Tramer et al. 2020).”
Better Diffusion Models Further Improve Adversarial Training (2023, background): “For standard training, label smoothing (LS) (Szegedy et al., 2016) improves standard generalization and alleviates the overconfidence problem (Hein et al., 2019), but it cannot prevent adaptive attacks from evading the model (Tramèr et al., 2020).”
Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models (2020, methodology): “It was shown in [1, 25] that existing preprocessing defenses can be totally broken or dramatically weakened by simple adjustments to the standard PGD attack, namely the EOT and BPDA techniques.”
On the Robustness of Vision Transformers to Adversarial Examples (2021, background): “Overall, CNNs have an expansive body of literature related to adversarial attacks [6, 11, 10] and defenses [23, 5, 34].”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “[157] also show that thirteen different defenses that actually account for adaptive attack strategies can also be broken.”
Backdoor Learning: A Survey (2020, background): “Such optimization in adversarial attacks requires multiple queries [160]–[162].”
Improving Robustness using Generated Data (2021, background): “Unfortunately, these techniques are not as robust as they seem and are broken by adaptive attacks [2, 16, 66].”
Bag of Tricks for Adversarial Training (2020, methodology): “Unfortunately, combing LS with standard training cannot prevent the models from evaded by adaptive attacks (Tramer et al., 2020) or larger iteration steps (Summers & Dinneen, 2018).”