Adversarial Examples Are a Natural Consequence of Test Error in Noise

By Nic Ford, Justin Gilmer, Nicholas Carlini, Ekin Dogus Cubuk. 2019

343 citations recorded by the source corpus.

Over the last few years, the phenomenon of adversarial examples --- maliciously constructed inputs that fool trained machine learning models --- has captured the attention of the research community, especially when the adversary is restricted to small modifications of a correctly handled input. Less surprisingly, image classifiers also lack human-level performance on randomly corrupted images, such as images with additive Gaussian noise. In this paper we provide both empirical and theoretical evidence that these are two manifestations of the same underlying phenomenon, establishing close connections between the adversarial robustness and corruption robustness research programs. This suggests that improving adversarial robustness should go hand in hand with improving performance in the presence of more general and realistic image corruptions. Based on our results we recommend that future adversarial defenses consider evaluating the robustness of their methods to distributional shift with benchmarks such as Imagenet-C.

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Molecular Biology, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Molecular Biology

In Safety, Robustness & Interpretability

Citations in the corpus

Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, methodology): “We make sure to make the distinction between in memory corruptions, for which we provide custom fast gpu implementations, and on disk corruptions, for which we use the publicly available ImageNetC dataset, since it was reported in [31] that jpeg compression can have a significant impact on model acc”
Certified Adversarial Robustness via Randomized Smoothing (2019, methodology): “, 2018b; Ford et al., 2019), and have suggested that Gaussian data augmentation could supplement or replace adversarial training (Zantedeschi et al., 2017; Kannan et al., 2018). Smilkov et al. (2017) observed that averaging a classifier’s input gradients over Gaussian corruptions of an image yields ”
A Fourier Perspective on Model Robustness in Computer Vision (2019, methodology): “As an example, in [10] it was observed that Gaussian data augmentation and adversarial training improve robustness to noise and blurring corruptions on the CIFAR-10-C and ImageNet-C common corruption benchmarks [17], while significantly degrading performance on the fog and contrast corruptions.”
On Evaluating Adversarial Robustness (2019, background): “– Add Gaussian noise of increasingly large standard deviation (Ford et al., 2019).”
Learning Data Augmentation Strategies for Object Detection (2019, methodology): “Instead of cropping to focus on parts of the image, some methods randomly erase or add noise to patches of images for improved accuracy [9, 53, 13], robustness [50, 12], or both [29].”
Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation (2019, methodology): “This metric is correlated with mCE [13], so it ensures model rosbustness is generally useful beyond Gaussian corruptions.”
Robustness properties of Facebook's ResNeXt WSL models (2019, background): “Given the large gains in robustness to common image corruptions and perturbations and the concomitant gains in adversarial robustness observed in the WSL models, our results are consistent with this prediction of Gilmer et al. (2019). The adversarial robustness of the WSL models, however, declined r”
Certified Adversarial Robustness with Additive Noise (2018, background): “Later, [23] showed adversarial robustness requires reducing error rates to essentially zero under large additive noise.”
Affinity and Diversity: Quantifying Mechanisms of Data Augmentation (2020, methodology): “In contrast, researchers have also successfully used more generic transformations that are less domain-speciﬁc, such as Gaussian noise (Ford et al., 2019; Lopes et al., 2019), input dropout (Srivastava et al., 2014), erasing random patches of the training samples during training (DeVries & Taylor,…”
Does Data Augmentation Lead to Positive Margin? (2019, background): “Recently, (Ford et al., 2019) studied the use of additive Gaussian DA in ensuring robustness of learned classiﬁers.”
Label-Only Membership Inference Attacks (2020, background): “Again, our intuition stems from linear models: a point’s distance to the boundary is directly related to the model’s accuracy when the point is perturbed by isotropic Gaussian noise [49].”
Stateful Detection of Black-Box Adversarial Attacks (2019, background): “If the adversary were content with finding any image that is misclassified, then an adversary could simply generate random candidate inputs and wait until a test error randomly occurred [16].”
Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2021, background): “In this context, the literature recommends that future adversarial defenses should consider evaluating the robustness of their methods to distributional shifts [81].”
RandAugment: Practical data augmentation with no separate search (2019, methodology): “Some methods randomly erase or add noise to patches of images for increased validation accuracy [8, 54], robustness [45, 51, 11], or both [31].”
The Origins and Prevalence of Texture Bias in Convolutional Neural Networks (2019, background): “Recent work has shown that data augmentation can also improve out-of-distribution robustness [33, 102, 65, 45, 78].”