Variational Approaches for Auto-Encoding Generative Adversarial Networks

By Mihaela Rosca, Balaji Lakshminarayanan, David Warde-Farley, Shakir Mohamed. 2017

282 citations recorded by the source corpus.

Auto-encoding generative adversarial networks (GANs) combine the standard GAN algorithm, which discriminates between real and model-generated data, with a reconstruction loss given by an auto-encoder. Such models aim to prevent mode collapse in the learned generative model by ensuring that it is grounded in all the available training data. In this paper, we develop a principle upon which auto-encoders can be combined with generative adversarial networks by exploiting the hierarchical structure of the generative model. The underlying principle shows that variational inference can be used a basic tool for learning, but with the in- tractable likelihood replaced by a synthetic likelihood, and the unknown posterior distribution replaced by an implicit distribution; both synthetic likelihoods and implicit posterior distributions can be learned using discriminators. This allows us to develop a natural fusion of variational auto-encoders and generative adversarial networks, combining the best of both these methods. We describe a unified objective for optimization, discuss the constraints needed to guide learning, connect to the wide range of existing work, and use a battery of tests to systematically and quantitatively assess the performance of our method.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Generative Models

Citations in the corpus

Autoregressive Quantile Networks for Generative Modeling (2018, methodology): “First, the α -GAN proposed by Rosca et al. (2017).”
Large Scale Adversarial Representation Learning (2019, background): “Other hybrid approaches like AGE [31] and α-GAN [27] add an encoder to stabilize GAN training.”
Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step (2017, methodology): “We note that the experimental results on real data for the non-saturating GAN and for the Improved Wasserstein GAN (WGAN-GP) are quoted with permission from an earlier publication by Rosca et al. [20].”
Multimodal Unsupervised Image-to-Image Translation (2018, methodology): “Latent Distribution Matching For image generation, existing works on combining auto-encoders and GANs need to match the encoded latent distribution with the latent distribution the decoder receives at generation time, using either KLD loss [15, 40] or adversarial loss [17, 42] in the latent space.”
Population Based Training of Neural Networks (2017, methodology): “Eval We evaluate the current model by a variant of the Inception score proposed by Salimans et al. (2016) computed from the outputs of a pretrained CIFAR classiﬁer (as used in Rosca et al. (2017)) rather than an ImageNet classiﬁer, to avoid directly optimising the ﬁnal performance metric.”
Recent Advances in Autoencoder-Based Representation Learning (2018, background): “Speciﬁcally, Larsen et al. [82], Rosca et al. [83] combine VAE with standard GAN [34], and Tschannen et al. [84] equip AAE/WAE with a Wasserstein GAN loss [85].”
Generating Natural Adversarial Examples (2017, background): “A number of further improvements to the GAN framework have been introduced (Salimans et al., 2016; Arjovsky & Bottou, 2017; Gulrajani et al., 2017; Rosca et al., 2017) that we discuss in Section 6.”
Distribution Matching in Variational Inference (2018, background): “This leads to the fusion of VAEs and GANs, of which we develop a deeper understanding by examining existing and new VAE-GAN hybrids [25, 30, 32, 38, 41, 44].”
Disentangling Adversarial Robustness and Generalization (2018, methodology): “On FONTS, the manifold is known by construction; in the other cases, the class manifolds have been approximated using VAE-GANs [51, 75].”
Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting (2019, methodology): “In generative modeling, estimating these ratios via binary classifiers is frequently used for defining learning objectives [11, 35].”
HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models (2019, methodology): “Even widely used metrics, such as Inception Score [52] and Fréchet Inception Distance [23], have been discredited for their application to non-ImageNet datasets [3, 48, 8, 46].”
Learning Data Augmentation for Brain Tumor Segmentation with Coarse-to-Fine Generative Adversarial Networks (2018, background): “They have also proven successful in a wide variety of applications such as image generation [1,8], image manipulation [13] and image inpainting [11].”
Hierarchical Adversarially Learned Inference (2018, methodology): “…for a given input x at the l -th hierarchical level is given by (5) Contrary to models that try to merge autoencoders and adversarial models, e.g. Rosca et al. (2017); Larsen et al. (2015), HALI does not require any additional terms in its loss function in order to minimize the above…”
How Generative Adversarial Networks and Their Variants Work (2017, methodology): “2) and auto-encoder reconstruction [14, 107, 128] (detailed in Section 3.”
Adversarial Variational Optimization of Non-Differentiable Simulators (2017, methodology): “, 2016), α-GAN (Rosca et al., 2017), AVB (Mescheder et al.”
How Generative Adversarial Networks and Its Variants Work : An Overview of GAN (2017, methodology): “[35]: Standard GAN, [73]: LSGAN, [138]: EBGAN, [6]: WGAN, [97]: Improved WGAN, [80]: McGAN, [10]: CramerGAN, [65]: MMDGAN, [79]: FisherGAN, [76]: Unrolled GAN, [14]: MRGAN, [59]: DRAGAN, [77]: Conditional GAN, [16]: InfoGAN, [106]: ss-InfoGAN, [27]: ALI, [25]: BiGAN, [116]: AGE, [58]: ARAE, [61]: VA”
Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55 (2017, background): “We extend α -GAN[15] to the task of 3D reconstruction from a single image.”