Evaluating and Understanding the Robustness of Adversarial Logit Pairing

By Logan Engstrom, Andrew Ilyas, Anish Athalye. 2018

145 citations recorded by the source corpus.

We evaluate the robustness of Adversarial Logit Pairing, a recently proposed defense against adversarial examples. We find that a network trained with Adversarial Logit Pairing achieves 0.6% accuracy in the threat model in which the defense is considered. We provide a brief overview of the defense and the threat models/claims considered, as well as a discussion of the methodology and results of our attack, which may offer insights into the reasons underlying the vulnerability of ALP to adversarial attack.

Subfields: Artificial Intelligence, Molecular Biology, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (2019, background): “Like nearly all other adversarial defenses, ALP was bypassed and has unclear value as an adversarial defense going forward (Engstrom et al., 2018), yet this is not a decisive reason dismiss it.”
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty (2019, methodology): “While techniques such as Cutout (Devries & Taylor, 2017) have not been demonstrated to help on the ImageNet scale, and while few have had success training adversarially robust models on ImageNet (Engstrom et al., 2018), other techniques such as Stylized ImageNet have been demonstrated to help on Ima”
Adversarial Training for Free! (2019, background): “Reference Engstrom et al. [2018] argues that gradient masking adds little security.”
Feature Denoising for Improving Adversarial Robustness (2018, critical): “ALP [10] was only evaluated against 10-iteration PGD attacks (Figure 6), and its claimed robustness is subject to controversy [5].”
Conditional variance penalties and domain shift robustness (2017, neutral): “Several empirical studies have shown mixed results regarding the performance on ∞ perturbations (Engstrom et al. 2018; Mosbach et al. 2018), so far this setting has not been analyzed theoretically and hence it is an open question whether a CoRetype penalty constitutes an effective defense against ad”
Using Pre-Training Can Improve Model Robustness and Uncertainty (2019, background): “First, nearly all adversarial defenses fail, and even some adversarial training methods can fail too (Engstrom et al., 2018).”
Theoretically Principled Trade-off between Robustness and Accuracy (2019, methodology): “We also compare TRADES with Adversarial Logit Pairing (ALP) (Kannan et al., 2018; Engstrom et al., 2018).”
On Evaluating Adversarial Robustness (2019, background): “More concerning than the fact that progress is slow is the fact that most proposed defenses are quickly shown to have performed incorrect or incomplete evaluations (Carlini & Wagner, 2016; 2017c; Brendel & Bethge, 2017; Carlini & Wagner, 2017a; He et al., 2017; Carlini & Wagner, 2017b; Athalye et al”
Unlabeled Data Improves Adversarial Robustness (2019, background): “A previous heuristic defense [20] against attacks with = 12/255 was subsequently broken [14].”
Fast is better than free: Revisiting adversarial training (2020, background): “…the history of adversarial examples is littered with adversarial defenses (Papernot et al., 2016; Lu et al., 2017; Kannan et al., 2018; Tao et al., 2018) which were ultimately defeated by stronger attacks (Carlini & Wagner, 2016; 2017b; Athalye et al., 2017; Engstrom et al., 2018; Carlini, 2019).”
Robustness properties of Facebook's ResNeXt WSL models (2019, background): “Engstrom et al. (2019) recently showed that these properties are signatures of adversarially non-robust models (robust models yield perceptually meaningful and heterogeneous maximizing images for different units and the maximizing images are much less dependent on the seed image).”
Logit Pairing Methods Can Fool Gradient-Based Attacks (2018, methodology): “In additional experiments (see Tables 4 and 5 in the appendix) we conﬁrm the hypothesis of [Engstrom et al., 2018] that adversarial training using untargeted PGD attack leads to improved adversarial accuracy.”
Adversarial Robustness Against the Union of Multiple Perturbation Models (2019, background): “However, as time goes by, the original robustness claims of these defenses typically don’t hold up to more advanced adversaries or more thorough attacks [Carlini and Wagner, 2017, Engstrom et al., 2018, Mosbach et al., 2018].”
Stateful Detection of Black-Box Adversarial Attacks (2019, background): “For example, services such as Clarifai [13] and Google Cloud Vision AI [19] offer image classification APIs where users can submit images and receive only the label of that image.”
Adversarial Attacks and Defenses in Deep Learning (2020, methodology): “[64] evaluates the robustness of an ALP-trained ResNet and discovers that the ResNet only achieves 0.”
Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples (2019, background): “However, ALP suffers from loss of robustness for in-distribution adversarial examples when the number of attack iterations is increased [38].”
Certified Adversarial Robustness with Additive Noise (2018, background): “Subsequent work has shown ALP fails to obtain adversarial robustness [35].”
Testing Robustness Against Unforeseen Adversaries (2019, background): “Many previous implementations of adversarial training fell prone to gradient masking [28, 30], with apparently successful training occurring only recently [3, 4].”
Recent Advances in Adversarial Training for Adversarial Robustness (2021, background): “But ALP initially is not useful due to the wrong formulation of adversarial training objectives [Engstrom et al. , 2018].”
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples (2020, background): “…in the literature (Papernot et al., 2015; Lu et al., 2017; Kannan et al., 2018; Tao et al., 2018; Zhang & Wang, 2019) were broken by stronger adversaries (Carlini & Wagner, 2016; 2017b; Athalye & Sutskever, 2017; Engstrom et al., 2018; Carlini, 2019; Uesato et al., 2018; Athalye et al., 2018).”