Generative Adversarial Parallelization

By Daniel Jiwoong Im, He Ma, Chris Dongjoo Kim, Graham W. Taylor. 2016

40 citations recorded by the source corpus.

Generative Adversarial Networks have become one of the most studied frameworks for unsupervised learning due to their intuitive formulation. They have also been shown to be capable of generating convincing examples in limited domains, such as low-resolution images. However, they still prove difficult to train in practice and tend to ignore modes of the data generating distribution. Quantitatively capturing effects such as mode coverage and more generally the quality of the generative model still remain elusive. We propose Generative Adversarial Parallelization, a framework in which many GANs or their variants are trained simultaneously, exchanging their discriminators. This eliminates the tight coupling between a generator and discriminator, leading to improved convergence and improved coverage of modes. We also propose an improved variant of the recently proposed Generative Adversarial Metric and show how it can score individual GANs or their collections under the GAP model.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Tengyu Ma
2. Sanjeev Arora
3. Torsten Hoefler
4. Yoshua Bengio
5. Tal Ben-Nun
6. Dacheng Tao
7. Yuanzhi Li
8. Yi Zhang

In Generative Models

1. Tal Ben-Nun
2. Torsten Hoefler
3. Graham W. Taylor
4. He Ma
5. Kristin Branson
6. Sanjeev Arora
7. Yi Zhang
8. Rong Ge

Citations in the corpus

• Generalization and Equilibrium in Generative Adversarial Nets (GANs) (2017, background): “…of the original idea, leading to more stable training and more realistic generative models in practice (see Odena et al. [2016], Huang et al. [2017], Radford et al. [2016], Tolstikhin et al. [2017], Salimans et al. [2016], Jiwoong Im et al. [2016], Durugkar et al. [2016] and the reference therein).”
• Demystifying Parallel and Distributed Deep Learning (2018, methodology): “zation[5,35,50,51,55,63,87,90,109,132,151,202,210,237,265] Sparsification[3,32,67,158,206,214,224] Other Methods[41,106,111,129,150,243,254] Training Distribution Model Consolidation Ensemble Learning[114,148,217] Knowledge Distillation[10,96] Model Averaging: Direct[20,33,168,266], Elastic[121,155,”
• Quantitatively Evaluating GANs With Divergences Proposed for Training (2018, methodology): “The Generative Adversarial Metric (Im et al., 2016a) measures the relative performance of two GANs by measuring the likelihood ratio of the two models.”