Wasserstein GAN
By Martín Arjovsky, Soumith Chintala, Soumith Chintala, Léon Bottou, L. Eon Bottou. 2017
5,178 citations recorded by the source corpus.
We introduce a new algorithm named WGAN, an alternative to traditional GAN training. In this new model, we show that we can improve the stability of learning, get rid of problems like mode collapse, and provide meaningful learning curves useful for debugging and hyperparameter searches. Furthermore, we show that the corresponding optimization problem is sound, and provide extensive theoretical work highlighting the deep connections to other distances between distributions.
Subfields: Molecular Biology, Radiology, Nuclear Medicine and Imaging, Generative Models
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Molecular Biology
- Tommi S. Jaakkola
- Regina Barzilay
- Song-Chun Zhu
- Ying Nian Wu
- Qiang Liu
- Yilun Du
- Stefano Ermon
- Igor Mordatch
In Radiology, Nuclear Medicine and Imaging
In Generative Models
- Stefano Ermon
- Jun-Yan Zhu
- Yoshua Bengio
- Richard Zhang
- Eli Shechtman
- Jiaming Song
- Arash Vahdat
- Jan Kautz
Citations in the corpus
- Evaluating Lossy Compression Rates of Deep Generative Models (2020, background): “For this reason, there has been much interest in Wasserstein distance as a criterion for generative models, since the measure exploits precisely this metric structure (Arjovsky et al., 2017; Gulrajani et al., 2017; Salimans et al., 2018).”
- Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling (2020, methodology): “The standard non-saturating training objective for the generator and discriminator is defined as: Wassertein GANs (WGAN) [14] are a special family of GAN models.”
- Learning Descriptor Networks for 3D Shape Synthesis and Analysis (2018, background): “It is also a generalized version of herding [27] and is related to [1].”
- Generalizing Hamiltonian Monte Carlo with Neural Networks (2017, background): “…can only indirectly measure the quality of the generated sample using adversarial training, which is notoriously unstable, suffers from “mode collapse” (where only a portion of a target distribution is covered), and often requires objective modification to train in practice (Arjovsky et al., 2017).”
- Learning Generative ConvNets via Multi-grid Modeling and Sampling (2017, methodology): “Within the same setting, we compare the learned features of the multi-grid method with the single-grid method, persistent CD [42], one-step CD [10], Wasserstein GAN [1], deep directed generative models [16] and DCGAN [37].”
- A-NICE-MC: Adversarial Training for MCMC (2017, methodology): “All the models are trained with the gradient penalty objective for Wasserstein GANs [17, 15], where = 1/3, B = 4 and M = 3.”
- D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, background): “Moreover, the GAN training procedure can be unstable [9, 8, 60], lack a informative objective for measuring progress [4], and struggle with discrete data [98].”
- Denoising Diffusion Implicit Models (2020, background): “However, GANs require very specific choices in optimization and architectures in order to stabilize training (Arjovsky et al., 2017; Gulrajani et al., 2017; Karras et al., 2018; Brock et al., 2018), and could fail to cover modes of the data distribution (Zhao et al., 2018).”
- VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, neutral): “Note that this approach is similar to the generator update in training Wasserstein GANs (Arjovsky et al., 2017).”
- EdgeConnect: Generative Image Inpainting with Adversarial Edge Learning (2019, methodology): “Spectral normalization was chosen over Wasserstein GAN (WGAN), [1] as we found that WGAN was several times slower in our early tests.”
- Improved Training of Wasserstein GANs (2017, methodology): “No critic batch normalization Most prior GAN implementations [21, 22, 2] use batch normalization in both the generator and the discriminator to help stabilize training, but batch normalization changes the form of the discriminator’s problem from mapping a single input to a single output to mapping f”
- GAIT: A Geometric Approach to Information Theory (2019, background): “Several studies (Arjovsky et al., 2017; Genevay et al., 2018; Salimans et al., 2018) have exhibited how divergences which induce a weak topology constitute learning signals with useful gradients.”
- High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs (2017, background): “GANs enable a wide variety of applications such as image generation [1, 41, 60], representation learning [44], image manipulation [62], object detection [32], and video applications [37, 50, 52].”
- Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step (2017, methodology): “For the parallel lines thought experiment [2] (see Figure 1), the main problem with the Jensen-Shannon divergence is that it is parameterized in terms of the density function, and the two density functions have no support in common.”
- Gradient descent GAN optimization is locally stable (2017, methodology): “Several methods have been proposed that introduce new objectives or architectures for improving the (practical and theoretical) stability of GAN optimization [Arjovsky et al., 2017, Poole et al., 2016].”
- The Numerics of GANs (2017, methodology): “…is orthogonal to strategies that try to make the GAN-game well-defined, e.g. by adding instance noise [21] or by using the Wasserstein-divergence [2, 9]: while these strategies try to ensure the existence of Nash-equilibria, our paper deals with their computation and the numerical difficulties…”