Dist-GAN: An Improved GAN Using Distance Constraints

By Ngoc-Trung Tran, Tuan-Anh Bui, Ngai-Man Cheung. 2018

97 citations recorded by the source corpus.

We introduce effective training algorithms for Generative Adversarial Networks (GAN) to alleviate mode collapse and gradient vanishing. In our system, we constrain the generator by an Autoencoder (AE). We propose a formulation to consider the reconstructed samples from AE as “real” samples for the discriminator. This couples the convergence of the AE with that of the discriminator, effectively slowing down the convergence of discriminator and reducing gradient vanishing. Importantly, we propose two novel distance constraints to improve the generator. First, we propose a latent-data distance constraint to enforce compatibility between the latent sample distances and the corresponding data sample distances. We use this constraint to explicitly prevent the generator from mode collapse. Second, we propose a discriminator-score distance constraint to align the distribution of the generated samples with that of the real samples through the discriminator score. We use this constraint to guide the generator to synthesize samples that resemble the real ones. Our proposed GAN using these distance constraints, namely Dist-GAN, can achieve better results than state-of-the-art methods across benchmark datasets: synthetic, MNIST, MNIST-1K, CelebA, CIFAR-10 and STL-10 datasets. Our code is published here (https://github.com/tntrung/gan) for research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Comprehensive Survey of Neural Architecture Search (2020, background): “Compared with the most advanced manual GANs [94–97], AutoGAN is very competitive.”
AutoGAN: Neural Architecture Search for Generative Adversarial Networks (2019, background): “In terms of inception score, AutoGAN is slightly next to Progressive GAN [26], and surpasses many latest strong competitors such as SN-GAN [42], improving MMD-GAN [56], Dist-GAN [54], MGAN [19], and WGAN-GP [13].”
InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learning (2020, background): “Consequently, recent years have seen concerted efforts [15, 23, 46, 54, 59, 60, 62] to mitigate these issues.”
Adversarial score matching and improved sampling for image generation (2020, methodology): “We note the similarities between these objective functions and those of an LSGAN adversarial autoencoder (Makhzani et al., 2015; Tolstikhin et al., 2017; Tran et al., 2018), with the distinction of using a denoising autoencoder H as opposed to a standard autoencoder.”
Improving GAN with neighbors embedding and gradient matching (2018, unknown): “However, the performance reported in (Tran, Bui, and Cheung 2018) is nearly saturated.”
Inclusive GAN: Improving Data and Minority Coverage in Generative Models (2020, methodology): “0261 Dist-GAN [47] 30.”
On Data Augmentation for GAN Training (2020, methodology): “It is worthnoting auto-encoder based methods [30], [35], [36], are likely good to mitigate catastrophic forgetting since the generator is regularized to resemble the real ones.”
An Improved Self-supervised GAN via Adversarial Training (2019, methodology): “Method CIFAR-10 STL-10 CIFAR-10 (R) STL-10 (R) CIFAR-10 (R) (10K-10K) GAN-GP [23] 37.7 - - - - WGAN-GP [23] 40.2 55.1 - - - SN-GAN [23] 25.5 43.2 21.70 ± .21 40.10 ± .50 19.73 SS-GAN [6] - - - - 15.65 Dist-GAN [31] 22.95 36.19 17.61 ± .30 28.50 ± .49 13.01 GN-GAN [30] 21.70 30.80 16.47 ± .28 - -”
Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game (2019, methodology): “We report the best FID attained in 300K iterations as in [45, 25, 42, 47].”
Few-shot Image Generation via Cross-domain Correspondence (2021, methodology): “A similar scheme has been employed for both uncon-ditional [27, 18] and conditional [19, 35] generation tasks to increase diversity in the generations.”