Self-supervised GAN: Analysis and Improvement with Multi-class Minimax Game

By Ngoc-Trung Tran, Viet-Hung Tran, Ngoc-Bao Nguyen, Linxiao Yang, Ngai-Man Cheung. 2019

68 citations recorded by the source corpus.

Self-supervised (SS) learning is a powerful approach for representation learning using unlabeled data. Recently, it has been applied to Generative Adversarial Networks (GAN) training. Specifically, SS tasks were proposed to address the catastrophic forgetting issue in the GAN discriminator. In this work, we perform an in-depth analysis to understand how SS tasks interact with learning of generator. From the analysis, we identify issues of SS tasks which allow a severely mode-collapsed generator to excel the SS tasks. To address the issues, we propose new SS tasks based on a multi-class minimax game. The competition between our proposed SS tasks in the game encourages the generator to learn the data distribution and generate diverse samples. We provide both theoretical and empirical analysis to support that our proposed SS tasks have better convergence property. We conduct experiments to incorporate our proposed SS tasks into two different GAN baseline models. Our approach establishes state-of-the-art FID scores on CIFAR-10, CIFAR-100, STL-10, CelebA, Imagenet $32\times32$ and Stacked-MNIST datasets, outperforming existing works by considerable margins in some cases. Our unconditional GAN model approaches performance of conditional GAN without using labeled data. Our code: \url{https://github.com/tntrung/msgan}

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learning (2020, background): “We conjecture this is due to the tendency for SSGAN to generate images that are easy to rotate [45], which sacrifices diversity and reduces FID when there are more classes.”
Data-Efficient Instance Generation from Instance Discrimination (2021, background): “[38] propose a multi-class minimax game to encourage the generator to produce diverse samples.”
Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis (2021, methodology): “Meanwhile, self-supervision on D has been shown to be an effective method to stabilize the GAN training as studied in Tran et al. (2019); Chen et al. (2019).”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “Extensions include training the discriminator to jointly determine rotation and real/fake to provide better feedback [221], and training the generator to trick the discriminator at both the real/fake and classification tasks [221].”
On Data Augmentation for GAN Training (2020, methodology): “We select one state-of-the-art GAN system recently published [46] and apply DAG.”