Adversarial Examples for Generative Models

By Jernej Kos, Ian S. Fischer, Dawn Song. 2017

289 citations recorded by the source corpus.

We explore methods of producing adversarial examples on deep generative models such as the variational autoencoder (VAE) and the VAE-GAN. Deep learning architectures are known to be vulnerable to adversarial examples, but previous work has focused on the application of adversarial examples to classification tasks. Deep generative models have recently become popular due to their ability to model input data distributions and generate realistic examples from those distributions. We present three classes of attacks on the VAE and VAE-GAN architectures and demonstrate them against networks trained on MNIST, SVHN and CelebA. Our first attack leverages classification-based adversaries by attaching a classifier to the trained encoder of the target generative model, which can then be used to indirectly manipulate the latent representation. Our second attack directly uses the VAE loss function to generate a target reconstruction image from the adversarial example. Our third attack moves beyond relying on classification or the standard loss for the gradient and directly optimizes against differences in source and target latent representations. We also motivate why an attacker might be interested in deploying such techniques against a target generative network.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Genetics, Generative Models, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Genetics

In Generative Models

In Safety, Robustness & Interpretability

Citations in the corpus

Certified Robustness to Adversarial Examples with Differential Privacy (2018, background): “Adversarial examples have also been demonstrated in other domains such as reinforcement learning [33] and generative models [32].”
The Robust Manifold Defense: Adversarial Training using Generative Models (2017, background): “This phenomenon is receiving a tremendous amount of recent attention (e.g. [31, 25, 48, 19, 22, 46] and references therein) for two good reasons: First a classiﬁer that can be easily fooled by non-perceivable noise poses a security threat in any real deployment.”
Robust Physical-World Attacks on Deep Learning Models (2017, background): “Recent work in computer vision, however, has demonstrated that DNNs are vulnerable to adversarial perturbations [3], [5], [8], [12], [13], [22], [23], [28].”
MagNet: A Two-Pronged Defense against Adversarial Examples (2017, background): “For example, feedforward convolutional classification networks [2], generative networks [14], and recurrent networks [27].”
Recent Progress on Generative Adversarial Networks (GANs): A Survey (2019, background): “Currently, there are already some works of attack on Convolutional Neural Network (CNN) [72]–[74], Recurrent Neural Networks (RNN) [75] and Deep Reinforcement Learning (DeepRL) [76], [77] and GANs [78].”
Adversarial Examples: Attacks and Defenses for Deep Learning (2017, background): “Attack” in [86]), VAE loss function (“LVAE Attck”), and distance between the original latent vector z and the modified encoded vector x (“Latent Attack,” similar to Tabacof et al.”
Generalization in anti-causal learning (2018, background): “To the best of our knowledge, adversarial examples have only been demonstrated in the anti-causal direction with a single exception for a generative variational auto-encoder (VAE) architecture (Kos et al., 2017).”
Towards neural networks that provably know when they don't know (2019, background): “Moreover, it has recently been discovered that VAEs, ﬂows and GANs also suffer from overconﬁdent likelihoods (Nalisnick et al., 2019; Hendrycks et al., 2019) far away from the data they are supposed to model as well as adversarial samples (Kos et al., 2017).”
Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation (2017, background): “One might think that generative adversarial neural networks should be resistant to this problem, but it has recently been shown [13] that they can also be attacked by adversarial manipulation of input images.”
Do Deep Generative Models Know What They Don't Know? (2018, background): “Adversarial attacks on generative models have been studied by (at least) Tabacof et al. (2016) and Kos et al. (2018), but these methods of attack require access to the model.”
Adversarially Robust Generalization Requires More Data (2018, background): “More recently, researchers have demonstrated the phenomenon under the name of adversarial examples in image classification [21, 51], question answering [28], voice recognition [12, 13, 49, 62], and other domains (for instance, see [2, 4, 14, 22, 25, 26, 32, 60]).”
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong (2017, background): “Research on other applications of neural networks has also encountered adversarial examples on different tasks beyond image classification, including deep policies in reinforcement learning and generative models [15, 14, 12, 2].”
Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey (2018, methodology): “[120] also explored methods for computing adversarial examples for deep generative models, e.”
Adversarial vulnerability for any classifier (2018, background): “We finally note that adversarial examples for generative models have recently been considered in [27]; our aim here is however different as our goal is to bound the robustness of classifiers when data comes from a generative model.”
Adversarial Transformation Networks: Learning to Generate Adversarial Examples (2017, background): “It would be interesting to explore the impact of ATNs on generative models, rather than just classiﬁers, similar to work in Kos et al. (2017).”
Distribution Matching in Variational Inference (2018, background): “Previous work has focused on ﬁnding adversarial examples as input to the VAE by ﬁnding points in data space that the VAE is unable to reconstruct (Kos et al., 2017).”
Understanding Membership Inferences on Well-Generalized Learning Models (2018, background): “attacks [17, 36] steal model parameters and structures; adversarial learning [13, 22, 27, 35] generates misleading examples that will be misclassified by the model; model inversion attacks [10, 14] infer the features of a record based on the model’s predictions on it; membership inference attacks [3”
Disentangling Adversarial Robustness and Generalization (2018, background): “Finally, adversarial examples have been applied to a wide variety of tasks, also beyond computer vision, e.g., [26, 18, 93, 47, 35, 54, 3, 16].”
DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars (2017, background): “These types of attacks are known to affect a broad spectrum of tasks such as image recognition [36, 39, 51, 54, 61, 62, 64, 65, 77], face detection/verification [74, 80], malware detection [27, 41, 53, 84], and text analysis [58, 66].”
Note on Attacking Object Detectors with Adversarial Stickers (2017, background): “ever, research has shown that deep networks are also vulnerable to digital manipulations of their inputs, resulting in a class of attacks known as ‘adversarial inputs’ [1, 3, 4, 6, 8].”