Improved Training of Wasserstein GANs

By Ishaan Gulrajani, Faruk Ahmed, Martín Arjovsky, Vincent Dumoulin, Aaron C. Courville. 2017

11,031 citations recorded by the source corpus.

Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. The recently proposed Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only low-quality samples or fail to converge. We find that these problems are often due to the use of weight clipping in WGAN to enforce a Lipschitz constraint on the critic, which can lead to undesired behavior. We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models over discrete data. We also achieve high quality generations on CIFAR-10 and LSUN bedrooms.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Yoshua Bengio
  2. Sergey Levine
  3. Mohit Bansal
  4. Quoc V. Le
  5. Jianfeng Gao
  6. Percy Liang
  7. Ruslan Salakhutdinov
  8. Dacheng Tao

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Trevor Darrell
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Bernt Schiele
  8. Andrea Vedaldi

In Generative Models

  1. Stefano Ermon
  2. Jun-Yan Zhu
  3. Richard Zhang
  4. Arash Vahdat
  5. Yoshua Bengio
  6. Eli Shechtman
  7. Jiaming Song
  8. Karsten Kreis

In Optimization & Training

  1. Yoshua Bengio
  2. Simon Lacoste-Julien
  3. Quoc V. Le
  4. Sergey Levine
  5. Stefano Ermon
  6. Frank Hutter
  7. Gauthier Gidel
  8. Pieter Abbeel

Citations in the corpus