Self-Supervised GANs via Auxiliary Rotation Loss

By Ting-Li Chen, Xiaohua Zhai, Marvin Ritter, Mario Lucic, Neil Houlsby. 2018

323 citations recorded by the source corpus.

Conditional GANs are at the forefront of natural image synthesis. The main drawback of such models is the necessity for labeled data. In this work we exploit two popular unsupervised learning techniques, adversarial training and self-supervision, and take a step towards bridging the gap between conditional and unconditional GANs. In particular, we allow the networks to collaborate on the task of representation learning, while being adversarial with respect to the classic GAN game. The role of self-supervision is to encourage the discriminator to learn meaningful feature representations which are not forgotten during training. We test empirically both the quality of the learned image representations, and the quality of the synthesized images. Under the same conditions, the self-supervised GAN attains a similar performance to state-of-the-art conditional counterparts. Finally, we show that this approach to fully unsupervised learning can be scaled to attain an FID of 23.4 on unconditional ImageNet generation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

On Robustness and Transferability of Convolutional Neural Networks (2020, background): “Auxiliary self-supervision [7, 71] can improve robustness to label noise and common corruptions [28].”
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark (2019, methodology): “Further, Zhai et al. (2019) show that using a small validation yields the same conclusions.”
Big Self-Supervised Models are Strong Semi-Supervised Learners (2020, background): “These contrastive learning based approaches learn representations in a discriminative fashion instead of a generative one as in [3, 49, 50, 41, 51].”
Generative Adversarial Networks (2021, neutral): “However, there are various obstacles and unresolved concerns that must be addressed in order to increase the performance and usability of GANs.”
A U-Net Based Discriminator for Generative Adversarial Networks (2020, background): “Other lines of work focused mainly on improving the discriminator by exploiting multiple [36, 13, 11] and multi-resolution [45, 42] discriminators, using spatial feedback of the discriminator [17], an auto-encoder architecture with the reconstructionbased feedback to the generator [52] or self-super”
Large Scale Adversarial Representation Learning (2019, methodology): “Other approaches include colorization [34, 35], motion segmentation [25], rotation prediction [9, 2], GAN-based discrimination [26, 2], and exemplar matching [6].”
ScaleNet: An Unsupervised Representation Learning Method for Limited Information (2023, methodology): “Since RotNet only affects the input images of the ConvNet, it can be combined with other architectures such as GAN [7] or be used for 3D structured datasets [26].”
Continual Learning on Noisy Data Streams via Self-Purified Replay (2021, background): “The pretext task that trains a model by predicting hidden information within the data includes patch orderings [17, 60], image impainting [62], colorization [87], and rotations [22, 13], to name a few.”
Co2L: Contrastive Continual Learning (2021, background): “To enjoy the beneﬁts of contrastive representation learning, each sample is augmented into two views following [11].”
InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learning (2020, methodology): “For CelebA, we follow [12] by taking 100K generator steps, each taken after 2 discriminator updates and with no learning rate decay.”
High-Fidelity Image Generation With Fewer Labels (2019, neutral): “While this performance is still considerably worse than that of B IG GAN this result is the current state-of-the-art in unsupervised image generation (Chen et al. (2019b) report an FID of 33 for unsupervised generation).”
Consistency Regularization for Generative Adversarial Networks (2019, neutral): “Previous work (Arjovsky & Bottou, 2017; Miyato et al., 2018a; Odena et al., 2017; Chen et al., 2019) has shown that interventions focused on the discriminator can mitigate stability issues.”
Image Augmentations for GAN Training (2020, methodology): “Besides simply adding augmentation to the data, some recent work [6, 42, 46] further added the regularization on top of augmentations to improve the model performance.”
Freeze the Discriminator: a Simple Baseline for Fine-Tuning GANs (2020, methodology): “, transferring knowledge of a welltrained source model [45, 32, 44], learning meta-knowledge for quick adaptation to a target domain [24, 47, 42], using an auxiliary task to facilitate training [7, 26, 48, 49], improving an inference procedure of suboptimal models [2, 39, 29, 38], using an expressiv”