Intriguing Properties of Adversarial Examples

By Ekin Dogus Cubuk, Barret Zoph, Sam Schoenholz, Quoc V. Le. 2017

90 citations recorded by the source corpus.

It is becoming increasingly clear that many machine learning classifiers are vulnerable to adversarial examples. In attempting to explain the origin of adversarial examples, previous studies have typically focused on the fact that neural networks operate on high dimensional data, they overfit, or they are too linear. Here we argue that the origin of adversarial examples is primarily due to an inherent uncertainty that neural networks have about their predictions. We show that the functional form of this uncertainty is independent of architecture, dataset, and training protocol; and depends only on the statistics of the logit differences of the network, which do not change significantly during training. This leads to adversarial error having a universal scaling, as a power-law, with respect to the size of the adversarial perturbation. We show that this universality holds for a broad range of datasets (MNIST, CIFAR10, ImageNet, and random data), models (including state-of-the-art deep networks, linear models, adversarially trained networks, and networks trained on randomly shuffled labels), and attacks (FGSM, step l.l., PGD). Motivated by these results, we study the effects of reducing prediction entropy on adversarial robustness. Finally, we study the effect of network architectures on adversarial sensitivity. To do this, we use neural architecture search with reinforcement learning to find adversarially robust architectures on CIFAR10. Our resulting architecture is more robust to white \emph{and} black box attacks compared to previous attempts.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, background): “Images (Szegedy et al., 2013; Cubuk et al., 2017) and audio (Carlini and Wagner, 2018) can easily be distorted with modiﬁcations that are imperceptible to humans.”
A Survey on Neural Architecture Search (2019, methodology): “As part of this we reconducted the experiments by Cubuk et al. (2018a) using their publicly available code.”
Adversarially Robust Generalization Requires More Data (2018, background): “For instance, contemporary approaches to architecture tuning do not yield better robust accuracies [13].”
Adversarial Logit Pairing (2018, background): “Model architecture plays a role in adversarial robustness (Cubuk et al., 2017), and models with higher capacities tend to be more robust (Kurakin et al., 2017a; Madry et al., 2017).”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “[91] argue that the ‘‘origin of adversarial examples is primarily due to an inherent uncertainty that neural networks have about their predictions’’.”
Learning Data Augmentation Strategies for Object Detection (2019, methodology): “The training setup for the RNN is similar to [47, 48, 4, 3].”
Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation (2019, background): “This agrees with the theoretical prediction from [5], which argued that increasing the conﬁdence of models would improve robustness, whereas label smoothing reduces the conﬁdence of predictions.”
Understanding and Mitigating the Tradeoff Between Robustness and Accuracy (2020, methodology): “We believe that since RST provides a complementary statistical perspective on the tradeoff, it can be combined with methods like IAT, NAS or AdvProp to see further gains in standard and robust errors.”
Techniques for Automated Machine Learning (2019, background): “One could make AutoDL robust to adversarial attack [18] and label noise [15].”
When NAS Meets Robustness: In Search of Robust Architectures Against Adversarial Attacks (2019, background): “Although the importance of architectures in adversarial robustness has emerged in the experiments of several previous work [28, 34, 19, 7], more comprehensive study on the role of network architectures in robustness remains needed.”
AutoAugment: Learning Augmentation Policies from Data (2018, methodology): “Our approach is inspired by recent advances in architecture search, where reinforcement learning and evolution have been used to discover model architectures from data [19, 21, 23, 33–41].”
On the Robustness of Semantic Segmentation Models to Adversarial Attacks (2017, methodology): “Figure 4 shows the results on the FGSM attack at ǫ = 8, for consistency with [26].”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “In [300], ‘inherent prediction uncertainty’ of neural networks is blamed for their adversarial vulnerability.”
AutoAugment: Learning Augmentation Strategies From Data (2019, methodology): “Our approach is inspired by recent advances in architecture search, where reinforcement learning and evolution have been used to discover model architectures from data [71, 4, 72, 7, 35, 13, 34, 46, 49, 63, 48, 9].”
Sensitivity and Generalization in Neural Networks: an Empirical Study (2018, background): “§ Certain measures of adversarial robustness have nevertheless been recently observed to correlate with generalization (Zahavy et al., 2018; Gilmer et al., 2018; Cubuk et al., 2018).”