Adversarial Feature Learning

By Jeff Donahue, Philipp Krahenbuhl, Trevor Darrell. 2016

1,916 citations recorded by the source corpus.

The ability of the Generative Adversarial Networks (GANs) framework to learn generative models mapping from simple latent distributions to arbitrarily complex data distributions has been demonstrated empirically, with compelling results showing that the latent space of such generators captures semantic variation in the data distribution. Intuitively, models trained to predict these semantic latent representations given data may serve as useful feature representations for auxiliary problems where semantics are relevant. However, in their existing form, GANs have no means of learning the inverse mapping -- projecting data back into the latent space. We propose Bidirectional Generative Adversarial Networks (BiGANs) as a means of learning this inverse mapping, and demonstrate that the resulting learned feature representation is useful for auxiliary supervised discrimination tasks, competitive with contemporary approaches to unsupervised and self-supervised feature learning.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Momentum Contrast for Unsupervised Visual Representation Learning (2019, methodology): “Adversarial methods for representation learning are explored in [15, 16].”
Self-Supervised Learning of Video-Induced Visual Invariances (2019, background): “Prior work has explored diverse self-supervision cues such as predicting the spatialcontext [12], colorization [71], equivariance to transformations [18, 44]; alongside unsupervised techniques such as clustering [6, 72], generative modelling [14, 32], and exemplar learning [15].”
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark (2019, background): “Sometimes generative models are also evaluated using linear classiﬁcation (Radford et al., 2016; Donahue et al., 2017; Dumoulin et al., 2017), or semi-supervised learning (Kingma et al., 2014; Narayanaswamy et al., 2017; Tschannen et al., 2018).”
Data-Efficient Image Recognition with Contrastive Predictive Coding (2019, background): “Generative adversarial models have also been harnessed for representation learning (Donahue et al., 2016), and largescale implementations have led to corresponding gains in linear classification accuracy (Donahue & Simonyan, 2019).”
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, background): “image datasets larger than ImageNet. In order to overcome the bottleneck, there have been recent efforts on visual representation learning using web-supervision [2,5,6,9,21, 23,24,27] or unsupervised [10,11,31,32,34,42,43] paradigms. However, most of these efforts are still are still exploratory in ”
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (2020, background): “Many of these approaches rely either on auto-encoding of images [24, 25, 26] or on adversarial learning [27], jointly modelling data and representation [28, 29, 30, 31].”
Contrastive Multiview Coding (2019, methodology): “As many previous unsupervised methods are evaluated with AlexNet [43] on ImageNet [16, 42, 17, 84, 54, 18, 85, 55, 22, 11, 83], we also include the the results of CMC using this network.”
Representation Learning with Contrastive Predictive Coding (2018, methodology): “The ImageNet dataset has been used to evaluate unsupervised vision models by many authors [28, 11, 35, 10, 29, 36].”
Unsupervised Feature Learning via Non-parametric Instance Discrimination (2018, methodology): “We compare our method with a randomly initialized network (as a lower bound) and various unsupervised learning methods, including self-supervised learning [2, 47, 27, 48], adversarial learning [4], and Exemplar CNN [3].”
Unsupervised Representation Learning by Predicting Image Rotations (2018, methodology): “…23.4 21.9 18.4 Colorization (Zhang et al., 2016a) 16.0 25.7 29.6 30.3 29.7 Jigsaw Puzzles (Noroozi & Favaro, 2016) 23.0 31.9 35.0 34.2 29.3 BIGAN (Donahue et al., 2016) 22.0 28.7 31.8 31.3 29.7 Split-Brain (Zhang et al., 2016b) 21.3 30.7 34.0 34.1 32.5 Counting (Noroozi et al., 2017) 23…”
Learning Features by Watching Objects Move (2016, methodology): “These models can be extended to produce good feature representations by training jointly with image encoders [10,11].”
Colorful Image Colorization (2016, methodology): “[16], and adding in color information increases performance to 35.”
A Style-Based Generator Architecture for Generative Adversarial Networks (2018, background): “While it is possible to add an extra network for this purpose [7, 11, 14], we want to avoid investing effort into a component that is not a part of the actual solution.”
Vector-quantized Image Modeling with Improved VQGAN (2021, background): “Many image generation models (Goodfellow et al., 2014; Kingma & Welling, 2014; Radford et al., 2016; Donahue et al., 2017; Higgins et al., 2017) have been studied for their capabilities in representation learning.”
D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation (2021, neutral): “Model family Explicit x→ z No prior hole Non-Adversarial (Enables SSL) (Better generation) (Stable training) VAE [52, 77], NF [28] 4 7 4 GAN [34] 7 4 7 BiGAN [30, 32] 4 4 7 DDIM [84] 7 4 4 D2C 4 4 4”
Dual Contradistinctive Generative Autoencoder (2020, neutral): “The subsequent development after GAN [55, 2, 17, 34, 15, 14, 13] has led to a great leap forward in building decoder-based generative models.”
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (2018, methodology): “Does the network have to be trained on classification? In Figure 4, we show model performance across a variety of unsupervised and self-supervised tasks, shown in green – generative modeling with BiGANs [13], solving puzzles [40], cross-channel prediction [63], and segmenting foreground objects from”
GAN Inversion: A Survey (2021, methodology): “Although some methods [31], [32], [33], [96] use additive”
Self-Supervised Learning of Pretext-Invariant Representations (2019, methodology): “, using sparse coding [58], adversarial training [12, 13, 50], autoencoders [49, 63, 76], or probabilistic versions thereof [67].”