Semi-Supervised Learning with Context-Conditional Generative Adversarial Networks

By Emily L. Denton, Sam Gross, Rob Fergus. 2016

165 citations recorded by the source corpus.

We introduce a simple semi-supervised learning approach for images based on in-painting using an adversarial loss. Images with random patches removed are presented to a generator whose task is to fill in the hole, based on the surrounding pixels. The in-painted images are then presented to a discriminator network that judges if they are real (unaltered training images) or not. This task acts as a regularizer for standard supervised training of the discriminator. Using our approach we are able to directly train large VGG-style networks in a semi-supervised fashion. We evaluate on STL-10 and PASCAL datasets, where our approach obtains performance comparable or superior to existing methods.

Subfields: Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Neural Discrete Representation Learning (2017, background): “Learning representations with continuous features have been the focus of many previous work [16, 39, 6, 9] however we concentrate on discrete representations [27, 33, 8, 28] which are potentially a more natural ﬁt for many of the modalities we are interested in. Language is inherently discrete,…”
Multi-task Self-Supervised Visual Learning (2017, background): “In the last few years, a considerable number of such tasks have been proposed [1, 2, 6, 7, 8, 17, 20, 21, 23, 25, 26, 27, 28, 29, 31, 39, 40, 42, 43, 46, 47], such as asking a neural network to colorize grayscale images, ﬁll in image holes, solve jigsaw puzzles made from image patches, or predict…”
ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring (2019, background): “SWWAE and CC-GAN results are from (Zhao et al., 2015) and (Denton et al., 2016).”
Self-Supervised Feature Learning by Learning to Spot Artifacts (2018, methodology): “[6] use the pretext task of inpainting and transfer the GAN discriminator features for classification.”
How Generative Adversarial Networks and Its Variants Work : An Overview of GAN (2017, methodology): “SSL-GAN [94] suggests an algorithm for semi-supervised learning with CC-GANs.”
How Generative Adversarial Networks and Their Variants Work (2017, background): “In addition, generative models can be the solution for various problems in the machine learning ﬁeld such as semisupervised learning [21, 67, 104, 115], which tries to address the lack of labeled data, and domain adaptation [2, 12, 47, 108, 111, 140], which leverages known knowledge for some tasks…”
Recovering Petaflops in Contrastive Semi-Supervised Learning of Visual Representations (2020, background): “Some non-contrastive approaches learn feature representations by relative patch prediction [12], by solving jigsaws [35], by applying and predicting image rotations [15], by inpainting or colorization [10, 37, 53, 54], by parameteric instance-discrimination [13], and sometimes by combinations thereo”
Stabilizing GAN Training with Multiple Random Projections (2017, background): “In particular, conditional variants of GANs have shown to be useful for tasks such as in-painting [2], and super-resolution [3].”
Deep Growing Learning (2017, background): “[3] introduced a simple semi-supervised learning approach to generate whole images based on in-painting using an adversarial loss.”
A Survey on Deep Semi-Supervised Learning (2021, background): “Overall, CatGAN [149] and CCGAN [150] extend the basic GAN by including additional information in the model, such as category information and in-painted images.”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “A research field where GANs are very successful is the application of generative models to semi-supervised learning [228], [229], as proposed but not shown in the first GANs paper [3].”
Supervision Accelerates Pre-training in Contrastive Semi-Supervised Learning of Visual Representations. (2020, background): “…approaches learn feature representations by relative patch prediction [13], by solving jigsaws [37], by applying and predicting image rotations [17], by inpainting or colorization [11, 39, 56, 57], by parametric instance-discrimination [14], and sometimes by combinations thereof [12, 29].”
Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks (2018, methodology): “…transfer GANs extract the content information from input data using auxiliary encoder networks (Yan et al., 2016; Perarnau et al., 2016; Antipov et al., 2017; Denton et al., 2016; Lu et al., 2017; Zhang et al., 2017; Isola et al., 2017; Taigman et al., 2017; Kim et al., 2017; Zhu et al., 2017).”
IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning (2018, methodology): “Like in [16], our model combines a conditional approach with a similar structure as of DCGAN [57].”
IntPhys: A Benchmark for Visual Intuitive Physics Reasoning (2019, methodology): “Results are detailed in Supplementary Materials (Tables 11, 12, 13, 14).”
Steering Self-Supervised Feature Learning Beyond Local Pixel Statistics (2020, methodology): “In contrast to prior SSL methods [25, 44, 7] , here we do not take the features from the networks that we used to learn the transformation (e.”