Explaining and Harnessing Adversarial Examples
By Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy. 2014
22,661 citations recorded by the source corpus.
Several machine learning models, including neural networks, consistently misclassify adversarial examples---inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence. Early attempts at explaining this phenomenon focused on nonlinearity and overfitting. We argue instead that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature. This explanation is supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Moreover, this view yields a simple and fast method of generating adversarial examples. Using this approach to provide examples for adversarial training, we reduce the test set error of a maxout network on the MNIST dataset.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Percy Liang
- Quoc V. Le
- Jianfeng Gao
- Sergey Levine
- Luke S. Zettlemoyer
- Ruslan Salakhutdinov
In Safety, Robustness & Interpretability
- Nicholas Carlini
- Matthias Hein
- Nicolas Papernot
- Bo Li
- Dawn Song
- Aleksander Madry
- Cho-Jui Hsieh
- Florian Tramèr
Citations in the corpus
- Learning Transferable Visual Models From Natural Language Supervision (2021, background): “, 2018), or adversarial attacks (Goodfellow et al., 2014) which are created by perturbing existing images in various ways.”
- Adversarial Training for Large Neural Language Models (2020, methodology): “To improve model robustness and withstand adversarial attacks, adversarial training has been proposed and studied extensively, predominantly in computer vision literature (Goodfellow et al., 2014; Madry et al., 2017).”
- Technical report on Conversational Question Answering (2019, methodology): “Adversarial training (Goodfellow et al., 2014) is a regularization method for neural networks to make the model more robust to adversarial noise and thereby improves its generalization ability.”
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks (2017, background): “Past work has observed that ReLU neural networks are locally almost linear (Goodfellow et al., 2015), which suggests that second derivatives may be close to zero in most cases, partially explaining the good perfor-Figure mance of the first-order approximation.”
- Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates (2018, methodology): “Additionally, we would like to explore the application of subword regularization for machine learning, including Denoising Auto En-coder (Vincent et al., 2008) and Adversarial Training (Goodfellow et al., 2015).”
- Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function (2019, positive): “The objective of adversarial training is to construct and give as input adversarial examples during model training procedure to make the model more robust to adversarial noise and thereby improving its generalization ability (Goodfellow, Shlens, and Szegedy 2014).”
- Adversarial Training Methods for Semi-Supervised Text Classification (2016, methodology): “Goodfellow et al. (2015) proposed to approximate this value by linearizing log p ( y | x ; ˆ θ ) around x .”
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning (2020, methodology): “Adversarial Training Adversarial machine learning is an active research area [63, 16, 5].”
- Do Image Classifiers Generalize Across Time? (2019, background): “In the machine learning community, model robustness has typically been analyzed on images perturbed by an adversary [11, 2], or by hand-designed strategies, such as rotations or blurs [7, 6, 14, 13].”
- Adversarial Examples Are Not Bugs, They Are Features (2019, methodology): “More broadly, previous work in the field tends to view adversarial examples as aberrations arising either from the high dimensional nature of the input space or statistical fluctuations in the training data [Sze+14; GSS15; Gil+18].”
- Making Convolutional Networks Shift-Invariant Again (2019, background): “These perturbations can be on pixels (Goodfellow et al., 2014a;b), a single pixel (Su et al., 2019), small deformations (Xiao et al., 2018), or even affine transformations (Engstrom et al., 2017).”
- Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects (2018, methodology): “One of the most effective methods for defending against OoD examples has been adversarial training [15], i.”
- Towards Robust Vision Transformer (2021, methodology): “For robustness evaluation, we test our RVT in three aspect: 1) for adversarial robustness, we test the adversarial examples generated by white-box attack algorithms FGSM [11] and PGD [26] on ImageNet-1K validation set.”
- CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features (2019, methodology): “Many researches have shown that deep models are easily fooled by small and unrecognizable perturbations on the input images, a phenomenon referred to as adversarial attacks [9, 38].”
- mixup: Beyond Empirical Risk Minimization (2017, methodology): “Other approaches perform data augmentation by producing and training on adversarial examples (Goodfellow et al., 2015).”
- LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop (2015, background): “The number of parameters in many deep models has exceeded the number of images on ImageNet, stretching the performance under the risk of the lack of bounded constraints [14, 10, 5].”
- CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders (2021, background): “• BigGan Optimization , in which images are produced using a pre-trained BigGAN generator.”