Improved generator objectives for GANs

By Ben Poole, Alexander A. Alemi, Jascha Narain Sohl-Dickstein, Anelia Angelova. 2016

73 citations recorded by the source corpus.

We present a framework to understand GAN training as alternating density ratio estimation and approximate divergence minimization. This provides an interpretation for the mismatched GAN generator and discriminator objectives often used in practice, and explains the problem of poor sample diversity. We also derive a family of generator objectives that target arbitrary $f$-divergences without minimizing a lower bound, and use them to train generative image models that target either improved sample quality or greater sample diversity.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Generative Models

Citations in the corpus

Improved Training of Wasserstein GANs (2017, background): “Nonlinearity (G) [ReLU, LeakyReLU, softplus(2x+2) 2 1, tanh] Nonlinearity (D) [ReLU, LeakyReLU, softplus(2x+2) 2 1, tanh] Depth (G) [4, 8, 12, 20] Depth (D) [4, 8, 12, 20] Batch norm (G) [True, False] Batch norm (D; layer norm for WGAN-GP) [True, False] Base filter count (G) [32, 64, 128] Base filte”
Unrolled Generative Adversarial Networks (2016, background): “minimizes the Jensen-Shannon divergence between the data distribution and the generator, and extensions generalize this to a wider class of divergences (Nowozin et al., 2016; Sonderby et al., 2016; Poole et al., 2016).”
Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks (2017, neutral): “This is similar to the approach taken by Poole et al. (Poole et al., 2016) to improve the GAN-objective.”
BEGAN: Boundary Equilibrium Generative Adversarial Networks (2017, critical): “Our observations seem to contradict those of [14] that diversity and quality were independent.”
On Variational Bounds of Mutual Information (2019, methodology): “This approach is similar to (Poole et al., 2016; Mescheder et al., 2017) but results in a bound instead of an unbounded estimate based on a Monte-Carlo approximation of the f -divergence.”
Do GANs actually learn the distribution? An empirical study (2017, methodology): “[3] proposed a method to trade-off between sample quality and sample diversity but they don’t provide a clear definition or a quantitative metric of sample diversity.”
GANs for Biological Image Synthesis (2017, background): “Another active area of research is improving the training objective function [33, 4, 37, 53, 1, 30, 42, 15].”
Improved Training with Curriculum GANs (2018, background): “This has led to a proliferation of work that focuses on improving the quality of the output of GANs by stabilizing the training procedure (Salimans et al., 2016; Poole et al., 2016; Warde-Farley and Bengio, 2016; Gulrajani et al., 2017; Karras et al., 2017).”
On variational lower bounds of mutual information (2018, methodology): “This approach is similar to Poole et al. (2016); Mescheder et al. (2017) where the JS critic is used in a Monte-Carlo approximation of the f -divergence, but here we plug into I NWJ to ensure we compute a lower bound (see Appendix C for details).”
Few-Shot Adaptation of Generative Adversarial Networks (2020, background): “Additionally, GANs can suffer from a lack of diversity, even when large training sets are used because the objective does not penalize the absence of outlier modes (Poole et al., 2016).”
Earth Mover’s Distance Minimization for Unsupervised Bilingual Lexicon Induction (2017, methodology): “Therefore, a lot of research efforts have been dedicated to the investigation into stabler training (Radford et al., 2015; Salimans et al., 2016; Nowozin et al., 2016; Metz et al., 2016; Poole et al., 2016; Arjovsky and Bottou, 2017), and the recently proposed Wasserstein GAN (Arjovsky et al.”