Improving MMD-GAN Training with Repulsive Loss Function

By Wei Wang, Yuan Sun, Saman Kumara Halgamuge. 2018

84 citations recorded by the source corpus.

Generative adversarial nets (GANs) are widely used to learn the data sampling process and their performance may heavily depend on the loss functions, given a limited computational budget. This study revisits MMD-GAN that uses the maximum mean discrepancy (MMD) as the loss function for GAN and makes two contributions. First, we argue that the existing MMD loss function may discourage the learning of fine details in data as it attempts to contract the discriminator outputs of real data. To address this issue, we propose a repulsive loss function to actively learn the difference among the real data by simply rearranging the terms in MMD. Second, inspired by the hinge loss, we propose a bounded Gaussian kernel to stabilize the training of MMD-GAN with the repulsive loss function. The proposed methods are applied to the unsupervised image generation tasks on CIFAR-10, STL-10, CelebA, and LSUN bedroom datasets. Results show that the repulsive loss function significantly improves over the MMD loss at no additional computational cost and outperforms other representative loss functions. The proposed methods achieve an FID score of 16.21 on the CIFAR-10 dataset using a single DCGAN network and spectral normalization.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dacheng Tao
2. Zhangyang Wang
3. Shiyu Chang
4. Xiaojun Chang
5. Po-Yao (Bernie) Huang
6. Changyou Chen
7. Jianfeng Gao
8. Xinyu Gong

In Computer Vision and Pattern Recognition

1. Zhangyang Wang
2. Dacheng Tao
3. Shiyu Chang
4. Zhenan Sun
5. Luc Van Gool
6. Shuicheng Yan
7. Jianfeng Gao
8. Ming-Hsuan Yang

In Generative Models

1. Zhangyang Wang
2. Shiyu Chang
3. Jianfeng Gao
4. Xinyu Gong
5. Dacheng Tao
6. Jie Gui
7. Jieping Ye
8. Ping Yu

In Optimization & Training

1. Dacheng Tao
2. Xinyu Gong
3. Jie Gui
4. Jieping Ye
5. Ping Yu
6. Xiaojun Chang
7. Changyou Chen
8. Zhenan Sun

Citations in the corpus

• Dual Contradistinctive Generative Autoencoder (2020, neutral): “8 - - Improving MMD GAN [64] 8.”
• A Comprehensive Survey of Neural Architecture Search (2020, background): “Compared with the most advanced manual GANs [95, 96, 97, 98], AutoGAN is highly competitive.”
• AutoGAN: Neural Architecture Search for Generative Adversarial Networks (2019, background): “In terms of inception score, AutoGAN is slightly next to Progressive GAN [26], and surpasses many latest strong competitors such as SN-GAN [42], improving MMD-GAN [56], Dist-GAN [54], MGAN [19], and WGAN-GP [13].”
• TransGAN: Two Transformers Can Make One Strong GAN (2021, methodology): “It could be viewed as a special case of non-local operations in the embedded Gaussian (Wang et al., 2018b), that captures long-range dependencies of pixels in image/video.”
• Feature Quantization Improves GAN Training (2020, methodology): “They further showed that several previous works on GAN can be written within the mean feature matching framework, including Wasserstein GAN (Arjovsky et al., 2017), MMD-GAN (Li et al., 2017b), and Improved GAN (Salimans et al., 2016).”
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, methodology): “The MMD has been widely used for GANs [170], [171], [172], [173], [174], [175], [176], [177].”