Constructing Unrestricted Adversarial Examples with Generative Models

By Yang Song, Rui Shu, Nate Kushman, Stefano Ermon. 2018

356 citations recorded by the source corpus.

Adversarial examples are typically constructed by perturbing an existing data point within a small matrix norm, and current defense methods are focused on guarding against this type of attack. In this paper, we propose unrestricted adversarial examples, a new threat model where the attackers are not restricted to small norm-bounded perturbations. Different from perturbation-based attacks, we propose to synthesize unrestricted adversarial examples entirely from scratch using conditional generative models. Specifically, we first train an Auxiliary Classifier Generative Adversarial Network (AC-GAN) to model the class-conditional distribution over data samples. Then, conditioned on a desired class, we search over the AC-GAN latent space to find images that are likely under the generative model and are misclassified by a target classifier. We demonstrate through human evaluation that unrestricted adversarial examples generated this way are legitimate and belong to the desired class. Our empirical results on the MNIST, SVHN, and CelebA datasets show that unrestricted adversarial examples can bypass strong adversarial training and certified defense methods designed for traditional adversarial attacks.

Subfields: Artificial Intelligence, Hardware and Architecture, Safety, Robustness & Interpretability, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Hardware and Architecture

In Safety, Robustness & Interpretability

In Signal Processing

Citations in the corpus

Natural Adversarial Examples (2019, background): “Several other forms of adversarial attacks have been considered in the literature, including elastic deformations [47], adversarial coloring [4, 26], and synthesis via generative models [3, 43] and evolutionary search [38], among others.”
The Robust Manifold Defense: Adversarial Training using Generative Models (2017, background): “Additionally, work by [Song et al., 2018] shows that generative models can be used to construct adversarial examples for classifiers, such that the classifier prediction is very different from a human prediction of the same image.”
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning (2020, methodology): “Constructing edited examples has also been used for adversarial robustness [31, 39].”
Excessive Invariance Causes Adversarial Vulnerability (2018, background): “…recent work argued that the perturbation-based approach is unrealistically restrictive and called for the need of generalizing the concept of adversarial examples to the unrestricted case, including any input crafted to be misinterpreted by the learned model (Song et al., 2018; Brown et al., 2018).”
On Evaluating Adversarial Robustness (2019, background): “However, at this time the authors need to completely 5See for example the work of Sharma & Chen (2017); Song et al. (2018) who explicitly step outside of the threat model of the original defenses to evaluate their robustness. switch their frame of mind and try as hard as possible to show their…”
Security and Privacy Issues in Deep Learning (2018, background): “ and Sutskever 2017] Momentum iFGSM [Dong et al. 2017] BPDA [Athalye et al. 2018] (Generative Model Attacks) Generating natural adversarial attack [Zhao et al. 2018] Constructing unrestricted attack [Song et al. 2018] Semantic adversarial attack [Joshi et al. 2019] Black-box Hacking smart machines [”
Robust Feature-Level Adversaries are Interpretability Tools (2021, methodology): “Others [55, 63, 29, 28, 60, 24] have used deep generative networks, and [69] aimed to create more semantically-understandable attacks by using an autoencoder with a “disentangled” embedding space.”
Cross-Domain Transferability of Adversarial Perturbations (2019, background): “[19] trains conditional generators to learn original data manifold and searches the latent space conditioned on the human recognizable target class that is mis-classified by a target classier.”
Achieving Robustness in the Wild via Adversarial Mixing With Disentangled Representations (2019, methodology): “Song et al. [34] and Xiao et al. [35] used a pre-trained AC-GAN [36] to generate perturbations; and they demonstrated that it is possible to generate semantically relevant perturbations for tasks such as M NIST , S VHN and C ELEB A. Lastly, Qiu et al. [37] have attempted to generate adversarial…”
Advances in adversarial attacks and defenses in computer vision: A survey (2021, background): “1This understanding of the term is slightly different from [43] and relates more to [44] that allows a clearer delineation between the unrestricted and conventional adversarial examples.”
Benchmarking Adversarial Robustness on Image Classification (2020, methodology): “In this paper, we study the well-defined and widely used lp norm threat models, although there also exist other threat models [58, 51, 18].”
The alignment problem from a deep learning perspective (2022, background): “One approach involves ﬁnding and training on unrestricted adversarial examples [Song et al., 2018] designed to prompt and penalize misaligned behavior.”
Interpreting Adversarially Trained Convolutional Neural Networks (2019, background): “, 2018) and GAN-based adversarial examples (Song et al., 2018).”
Adversarial Attacks and Defenses in Images, Graphs and Text: A Review (2019, methodology): “However, the work (Song et al., 2018) introduces a method to generate unrestricted adversarial examples.”
Universal Adversarial Attacks with Natural Triggers for Text Classification (2020, methodology): “Song et al. (2018) use a standard Auxiliary Classiﬁer GAN to synthesize images, which have different predictions between the auxiliary classi-ﬁer and the target classiﬁer.”
Adversarial Attacks and Defenses in Deep Learning (2020, unknown): “[31] train an auxiliary classifier GAN (AC-GAN) [32] to model the data distribution for each class.”