Exploring the space of adversarial images

By Pedro Tabacof, Eduardo Valle. 2015

200 citations recorded by the source corpus.

Adversarial examples have raised questions regarding the robustness and security of deep neural networks. In this work we formalize the problem of adversarial images given a pretrained classifier, showing that even in the linear case the resulting optimization problem is nonconvex. We generate adversarial images using shallow and deep classifiers on the MNIST and ImageNet datasets. We probe the pixel space of adversarial images using noise of varying intensity and distribution. We bring novel visualizations that showcase the phenomenon and its high variability. We show that adversarial images appear in large regions in the pixel space, but that, for the same task, a shallow classifier seems more robust to adversarial images than a deep convolutional network.

Subfields: Artificial Intelligence, Biophysics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

Citations in the corpus

AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation (2018, background): “, [3], [12], [14], [15], [17], [18], [29], [30], [32], [38], [41]).”
Universal Adversarial Perturbations (2016, background): “The robustness of image classiﬁers to structured and un-structured perturbations have recently attracted a lot of attention [19, 16, 20, 3, 4, 12, 13, 14].”
Learning with a Strong Adversary (2015, background): “Additional exploration by Tabacof & Valle (2015) has demonstrated that, for image classiﬁcation problems, adversarial images inhabit large ”adversarial pockets” in the pixel space.”
Measuring Neural Net Robustness with Constraints (2016, background): “(1) This definition formalizes the notion of robustness in [5, 6, 20].”
Adversarial Manipulation of Deep Representations (2015, methodology): “…have shown that deep neural networks (DNNs) for image classiﬁcation can be fooled, often using relatively simple methods to generate so-called adversarial images (Fawzi et al., 2015; Goodfellow et al., 2014; Gu & Rigazio, 2014; Nguyen et al., 2015; Szegedy et al., 2014; Tabacof & Valle, 2015).”
Robustness of classifiers: from adversarial to random noise (2016, background): “More recently, following the recent results on the instability of deep neural networks to worst-case perturbations [17], several works have provided explanations of the phenomenon [3, 5, 14, 18], and designed more robust networks [6, 8, 20, 13, 15, 12].”
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong (2017, background): “On the other hand, prior work has shown that larger models are actually easier to fool, with lower-distortion adversarial examples or better success at a fixed level of distortion [7, 21, 28, 4].”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, background): “Tabacof and Valle [25] generated adversarial examples for shallow and deep network classifiers on MNIST [10] and ImageNet [11] datasets and probed the pixel space of adversarial examples by using noise of varying distribution and intensity.”
ConvNets and ImageNet Beyond Accuracy: Explanations, Bias Detection, Adversarial Examples and Model Criticism (2017, background): “Such perturbed inputs, called adversarial examples, have recently aroused keen interest in the community [17, 42, 43, 7].”
Analysis of universal adversarial perturbations (2017, background): “classifiers to adversarial perturbations has been specifically studied, leading to theoretical as well empirical explanations of the phenomenon [9], [10], [11], [12].”
Houdini: Fooling Deep Structured Prediction Models (2017, background): “This ﬁnding has recently aroused keen interest in the community [12, 28, 32, 33].”
Retrieval-Augmented Convolutional Neural Networks for Improved Robustness against Adversarial Examples (2018, methodology): “We extensively test the retrieval-augmented convolutional network (RaCNN) on datasets with varying scales; CIFAR-10 (Krizhevsky & Hinton, 2009), SVHN (Netzer et al., 2011) as well as Ima-geNet (Deng et al., 2009), against ﬁve readily-available adversarial attacks including both white-box (FGSM, iFGS”
Robustness of classifiers to uniform $\ell_p$ and Gaussian noise (2018, background): “that deep neural networks are not robust to worst-case, or adversarial, perturbations. Several works have followed and attempted to provide explanations to the vulnerability [Goodfellow et al., 2015, Tabacof and Valle, 2016, Tanay and Grin, 2016, Sabour et al., 2016]. In particular, it was shown the”
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review (2017, unknown): “…2021 2014; Goodfellow et al., 2015; Gu & Rigazio, 2014; Maharaj, 2015; Zhao & Griffin, 2016; Uličný et al., 2016; Jin, Dundar, & Culurciello, 2016; Tabacof & Valle, 2016; Miyato, Maeda, Koyama, Nakae, & Ishii, 2016; Sabour, Cao, Faghri, & Fleet, 2016; Papernot, McDaniel, Xu, Jha, & Swami, 2016;…”
EagleEye: Attack-Agnostic Defense against Adversarial Inputs (Technical Report) (2018, background): “One line of work focuses on developing new attacks against DNNs [8, 14, 21, 33, 43], most of which attempt to find the minimum possible modifications to the input data to trigger the systems to misclassify.”
A study of the effect of JPG compression on adversarial images (2016, background): “Tabacof and Valle [TV15] demonstrated empirically that adversarial examples are not isolated points and the neural networks are more robust to random noise than adversarial noise.”
ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases (2017, background): “Such perturbed inputs, called adversarial examples, have recently aroused keen interest in the community [22,15,23,17].”