Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks

By Lars M. Mescheder, Sebastian Nowozin, Andreas Geiger. 2017

572 citations recorded by the source corpus.

Variational Autoencoders (VAEs) are expressive latent variable models that can be used to learn complex probability distributions from training data. However, the quality of the resulting model crucially relies on the expressiveness of the inference model. We introduce Adversarial Variational Bayes (AVB), a technique for training Variational Autoencoders with arbitrarily expressive inference models. We achieve this by introducing an auxiliary discriminative network that allows to rephrase the maximum-likelihood-problem as a two-player game, hence establishing a principled connection between VAEs and Generative Adversarial Networks (GANs). We show that in the nonparametric limit our method yields an exact maximum-likelihood assignment for the parameters of the generative model, as well as the exact posterior distribution over the latent variables given an observation. Contrary to competing approaches which combine VAEs with GANs, our approach has a clear theoretical justification, retains most advantages of standard Variational Autoencoders and is easy to implement.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Generative Models

Citations in the corpus

Self-Supervised Learning of Pretext-Invariant Representations (2019, methodology): “, using sparse coding [58], adversarial training [12, 13, 50], autoencoders [49, 63, 76], or probabilistic versions thereof [67].”
Scaling and Benchmarking Self-Supervised Visual Representation Learning (2019, background): “This can be modeled as maximizing the probability of reconstructing the input [43, 51, 67] and optionally estimating latent variables [29, 58] or using adversarial training [15, 44].”
The Numerics of GANs (2017, methodology): “Since their first appearance, GANs have been successfully applied to a variety of tasks, including image-to-image translation [12], image super-resolution [13], image in-painting [27] domain adaptation [26], probabilistic inference [14, 9, 8] and many more.”
Comparison of Maximum Likelihood and GAN-based training of Real NVPs (2017, methodology): “We show one additional usage of real NVPs for Adversarial Variational Bayes (Mescheder et al., 2017) evaluation in the appendix.”
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, background): “The Deep Learning community has recently proposed Generative Adversarial Networks (GANs) [29, 68, 70], which are still being intensively developed [3, 9, 31, 41, 54].”
Variational Autoencoders Pursue PCA Directions (by Accident) (2018, background): “One surprising aspect is that VAEs have been challenged on both of its own design functionalities, as generative models [14, 12] and as log-likelihood optimizers [30, 33].”
Disentangling by Factorising (2018, methodology): “In this work, we minimise divergence between two distributions over the latent space (as in e.g. (Mescheder et al., 2017)), which is typically much lower dimensional and the two distributions have overlapping support.”
Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems (2019, methodology): “To this end, several methods have been proposed to overcome these limitations, including more expressive likelihood models [46], more flexible variational approximations [37, 36, 41], as well as reformulations that aim to make the variational bound of equation 2 more tight [47, 48, 49, 50, 33].”
Deep Generative Dual Memory Network for Continual Learning (2017, background): “Another recent work by Mescheder et al. (2017) which improves the generative performance of VAEs by learning the posterior distribution implicitly (like in GANs), can also replace the VAE as the choice of generative model.”
Posterior Meta-Replay for Continual Learning (2021, neutral): “[49] L.”
Learning latent representations across multiple data domains using Lifelong VAEGAN (2020, methodology): “odels which would only train generator and inference models within a single optimization function [36] or learn an optimal coupling between generator and inference model by using adversarial learning [11,15,16,36,37,42,43,61]. The proposed algorithm contains two independent optimization paths, namel”
Continual learning with hypernetworks (2019, background): “Such flexibility could be exploited via the adversarial variational Bayes framework [20], which enables variational inference with implicit models.”
Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play (2017, background): “(Mescheder et al., 2017) demonstrate how adversarial loss terms can be combined with variational auto-encoders to permit more accurate density modeling.”
Learning in Implicit Generative Models (2016, methodology): “…example, Karaletsos (2016) use GAN-like techniques for inference in factor graphs, and since the central quantity of variational inference is a density ratio it is possible to propose a modiﬁed variational inference using implicit models, as discussed by Mescheder et al. (2017) and Huszár (2017).”
On Variational Bounds of Mutual Information (2019, methodology): “This approach is similar to (Poole et al., 2016; Mescheder et al., 2017) but results in a bound instead of an unbounded estimate based on a Monte-Carlo approximation of the f -divergence.”
ALICE: Towards Understanding Adversarial Learning for Joint Distribution Matching (2017, neutral): “, classification from the latent variable) [2, 3, 4, 5, 6, 7].”
VAE with a VampPrior (2017, background): “Last but not least, it would be interesting to utilize a normalizing flow [32], the hierarchical variational inference [31], ladder networks [36] or adversarial training [28] within the VampPrior VAE.”
Adversarial Feature Matching for Text Generation (2017, background): “Recent work considered combining autoencoders or variational autoencoders (Kingma & Welling, 2014) with GAN (Zhao et al., 2017; Larsen et al., 2016; Makhzani et al., 2015; Mescheder et al., 2017; Wang & Liu, 2016).”