Semi-Supervised Learning with Generative Adversarial Networks

By Augustus Odena. 2016

733 citations recorded by the source corpus.

We extend Generative Adversarial Networks (GANs) to the semi-supervised context by forcing the discriminator network to output class labels. We train a generative model G and a discriminator D on a dataset with inputs belonging to one of N classes. At training time, D is made to predict which of N+1 classes the input belongs to, where an extra class is added to correspond to the outputs of G. We show that this method can be used to create a more data-efficient classifier and that it allows for generating higher quality samples than a regular GAN.

Subfields: Computer Vision and Pattern Recognition, Generative Models, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

In Representation & Self-Supervised Learning

Citations in the corpus

S4L: Self-Supervised Semi-Supervised Learning (2019, background): “Many of the initial results on semi-supervised learning with deep neural networks were based on generative models such as denoising autoencoders [31], variational autoencoders [15] and generative adversarial networks [27, 33].”
Improved Techniques for Training GANs (2016, background): “In concurrent work, Odena [15] proposes to extend GANs to predict image labels like we do in Section 5, but without our feature matching extension (Section 3.”
Generative Adversarial Networks (2021, neutral): “Mode collapse, on the other hand, can occur when the Generator Network learns to create a restricted number of samples that fail to accurately reflect the diversity of the underlying data distribution.”
Score-based Generative Modeling in Latent Space (2021, background): “Furthermore, like other deep generative models, it can potentially be used also for non-generative tasks such as semi-supervised and representation learning [101, 102, 103].”
MixMatch: A Holistic Approach to Semi-Supervised Learning (2019, neutral): “, “transductive” models [14, 22, 21], graph-based methods [49, 4, 29], generative modeling [3, 27, 41, 9, 17, 23, 38, 34, 42], etc.”
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation (2017, background): “vided both the discriminator and generator with class information in order to generate samples conditioned on the class [19, 20, 18].”
Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis (2017, background): “It has also been empirically shown [2–4, 8] that adding auxiliary tasks seems to reduce such a tendency and improve convergence stability; side information is therefore often used as either additional information to the discriminator / generator [4, 5, 8], as a quantity to be reconstructed by the di”
Conditional Image Synthesis with Auxiliary Classifier GANs (2016, methodology): “Better models of natural images also improve performance in semi-supervised learning tasks (Kingma et al., 2014; Springenberg, 2015; Odena, 2016; Salimans et al., 2016) and reinforcement learning problems (Blundell et al.”
Temporal Ensembling for Semi-Supervised Learning (2016, methodology): “count the evaluations over multiple epochs. Generative Adversarial Networks (GAN) have been recently used for semi-supervised learning with promising results (Maaløe et al., 2016; Springenberg, 2016; Odena, 2016; Salimans et al., 2016). It 8 Published as a conference paper at ICLR 2017 Table 5: The ”
Learning Adversarially Fair and Transferable Representations (2018, methodology): “…for deep generative modeling was pop-ularized by Goodfellow et al. (2014) and applied to deep semi-supervised learning (Salimans et al., 2016; Odena, 2016) and segmentation (Luc et al., 2016), although similar concepts had previously been proposed for unsupervised and supervised learning…”
A Bayesian Data Augmentation Approach for Learning Deep Models (2017, background): “Variants of GAN have been proposed to improve the quality of the synthetic images [22, 3, 23, 24].”
High-Fidelity Image Generation With Fewer Labels (2019, neutral): “Both Salimans et al. (2016) and Odena (2016) train a discrimi-nator that classiﬁes its input into K + 1 classes: K image classes for real images, and one class for generated images.”
Machine Learning with Membership Privacy using Adversarial Regularization (2018, methodology): “Adversarial training has also been used for regularizing, hence generalizing, a model [11, 14, 29, 35, 36, 38].”
Data-Free Learning of Student Networks (2019, background): “Odena [18] further suggested that the tasks of discrimination and classification can improve each other.”
Discriminator Rejection Sampling (2018, methodology): “They have also been applied successfully to semi-supervised and unsupervised learning (Springenberg, 2015; Odena, 2016; Kumar et al., 2017), image editing (Yu et al., 2018; Ledig et al., 2017), and image style transfer (Zhu et al., 2017; Isola et al., 2017; Yi et al., 2017; Azadi et al., 2018).”
An empirical study on evaluation metrics of generative adversarial networks (2018, neutral): “…et al., 2017; Arjovsky et al., 2017; Gulrajani et al., 2017), they have also been innovatively applied in, for example, semi-supervised learning (Odena, 2016; Makhzani et al., 2015), image-to-image translation (Isola et al., 2016; Zhu et al., 2017), and simulated image reﬁnement (Shrivastava et…”
Triple Generative Adversarial Nets (2017, background): “Odena (2016) and Salimans et al. (2016) augment the categorical discriminator with one more class, corresponding to the fake data generated by the generator.”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “Odena [230] extends GANs to the semi-supervised learning by forcing the discriminator network to output class labels.”