VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning

By Akash Srivastava, Lazar Valkov, Chris Russell, Michael U Gutmann, Charles Sutton. 2017

740 citations recorded by the source corpus.

Deep generative models provide powerful tools for distributions over complicated manifolds, such as those of natural images. But many of these methods, including generative adversarial networks (GANs), can be difficult to train, in part because they are prone to mode collapse, which means that they characterize only a few modes of the true distribution. To address this, we introduce VEEGAN, which features a reconstructor network, reversing the action of the generator by mapping from data to noise. Our training objective retains the original asymptotic consistency guarantee of GANs, and can be interpreted as a novel autoencoder loss over the noise. In sharp contrast to a traditional autoencoder over data points, VEEGAN does not require specifying a loss function over the data, but rather only over the representations, which are standard normal by assumption. On an extensive set of synthetic and real world image datasets, VEEGAN indeed resists mode collapsing to a far greater extent than other recent GAN variants, and produces more realistic samples.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Generative Models

Citations in the corpus

Imperfect ImaGANation: Implications of GANs Exacerbating Biases on Facial Data Augmentation and Snapchat Selfie Lenses (2020, background): “…variants that seek to address mode-collapse, have proven to be worse than AdaGAN (Lala et al., 2018); such as WGAN (Arjovsky et al., 2017), VEEGAN (Srivastava et al., 2017) or Unrolled GAN (Metz et al., 2017) and either affect the quality of generated images, exacerbate the biases over latent…”
Multimodal Unsupervised Image-to-Image Translation (2018, neutral): “The style reconstruction loss Li recon is reminiscent of the latent reconstruction loss used in the prior works [11, 31, 44, 57].”
Learning latent representations across multiple data domains using Lifelong VAEGAN (2020, methodology): “Adversarial learning is performed in order to match either the data distribution [36], the latent variables distribution [42], or their joint distributions [10,15,16,37,43,49,61].”
InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learning (2020, background): “These works often focused on assessing the number of modes recovered by a GAN based on synthetic datasets [48], [51], [52].”
Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay (2019, background): “However, adversarial learning is known to require deliberate architecture design and selection of hyper-parameters [Roth et al., 2017], and can suffer from mode collapse [Srivastava et al., 2017].”
Adversarial score matching and improved sampling for image generation (2020, neutral): “…Modes (Max 1000) KL DCGAN (Radford et al., 2015b) 99.0 3.40 ALI (Dumoulin et al., 2016) 16.0 5.40 Unrolled GAN (Metz et al., 2016) 48.7 4.32 VEEGAN (Srivastava et al., 2017) 150.0 2.95 PacDCGAN2 (Lin et al., 2017b) 1000.0 0.06 WGAN-GP (Kumar et al., 2019; Gulrajani et al., 2017) 959.0 0.73 PresGAN…”
Adversarial Latent Autoencoders (2020, background): “For the latent space instead, the common practice is to set a desired target latent distribution, and then the encoder is trained to match it either by minimizing a divergence type of similarity [28, 41, 31, 47, 48], or by setting up an adversarial game [35, 6, 9, 49, 20].”
Mutual Information Neural Estimation (2018, background): “Number of captured modes and Kullblack-Leibler divergence between the training and samples distributions for DC-GAN (Radford et al., 2015), ALI (Dumoulin et al., 2016), Un-rolled GAN (Metz et al., 2017), VeeGAN (Srivastava et al., 2017), PacGAN (Lin et al., 2017). and reverse (decoder) models, respe”
Distribution Matching in Variational Inference (2018, methodology): “VEEGAN Srivastava et al. (2017) changes the model objective from matching the marginals p ∗ ( x ) and p θ ( x ) in data space to matching marginal distributions in latent space.”
Discriminator Rejection Sampling (2018, methodology): “05) arranged in a grid (Dumoulin et al., 2016; Srivastava et al., 2017; Lin et al., 2017).”
PacGAN: The Power of Two Samples in Generative Adversarial Networks (2017, background): “contrast with other approaches for mitigating mode collapse that require significant computational overhead and/or delicate hyperparameter selection [16], [18], [19], [20], [21].”
Pros and Cons of GAN Evaluation Measures (2018, methodology): “In each of the two panels, the first column shows real images, followed by the nearest image generated by DCGAN [64], ALI [20], Unrolled GAN [59], and VEEGAN [75], respectively.”
Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models (2017, methodology): “This line of reasoning has been explored to some extent in prior work that combine the objectives of prescribed latent variable models such as VAEs (maximizing an evidence lower bound on the data) with adversarial learning (Larsen et al. 2015; Mescheder, Nowozin, and Geiger 2017a; Srivastava et al. ”
Maximum Entropy Generators for Energy-Based Models (2019, methodology): “, 2016) and (Srivastava et al., 2017).”
Generative Adversarial Network Training is a Continual Learning Problem (2018, background): “One particularly well-known and common failure mode is mode collapse (Che et al., 2017; Srivastava et al., 2017): instead of producing samples sufﬁciently representing the true data distribution, the generator maps the entire latent space to a limited subset of the real data space.”
Generative Latent Flow: A Framework for Non-adversarial Image Generation (2019, background): “Although there have been multiple efforts to overcome the difficulties in training GANs, mainly by modifying the objective functions [22, 23, 21], researchers are also actively studying non-adversarial generative models that are known to be less affected by these issues.”
Evaluation of Mode Collapse in Generative Adversarial Networks (2018, methodology): “Here we review the most recent, widely used GANs such as Wasserstein GAN (WGAN) [8], VEEGAN [9], Unrolled GAN [7], and AdaGAN [10].”
Deep Neural Networks and Tabular Data: A Survey (2021, background): “Moreover, VeeGAN [139] is frequently used as a reference for tabular data generation [38], [130], [131].”