Discriminative Regularization for Generative Models

By Alex Lamb, Vincent Dumoulin, Aaron C. Courville. 2016

67 citations recorded by the source corpus.

We explore the question of whether the representations learned by classifiers can be used to enhance the quality of generative models. Our conjecture is that labels correspond to characteristics of natural data which are most salient to humans: identity in faces, objects in images, and utterances in speech. We propose to take advantage of this by using the representations from discriminative classifiers to augment the objective function corresponding to a generative model. In particular we enhance the objective function of the variational autoencoder, a popular generative model, with a discriminative regularization term. We show that enhancing the objective function in this way leads to samples that are clearer and have higher visual quality than the samples from the standard variational autoencoders.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

Citations in the corpus

Taming Transformers for High-Resolution Image Synthesis (2020, methodology): “To do so, we propose VQGAN, a variant of the original VQVAE, and use a discriminator and perceptual loss [36, 26, 35, 16] to keep good perceptual quality at increased compression rate.”
Generating Images with Perceptual Similarity Metrics based on Deep Networks (2016, background): “We found that adding a loss in the image space stabilizes training.”
Adversarially Learned Inference (2016, methodology): “Lamb et al. (2016) use the hidden layers of a pre-trained classiﬁer as auxiliary reconstruction losses to help the VAE focus on higher-level details when reconstructing.”
Neural Photo Editing with Introspective Adversarial Networks (2016, methodology): “Using the intermediate activations of a pre-trained discriminative neural network as features for comparing reconstructions to originals (Lamb et al., 2016) molliﬁes this effect, but requires labels in order to train the discriminative network in a supervised fashion.”
Stacked Generative Adversarial Networks (2016, background): “[31, 7] add a regularization term that encourages the reconstructed image to be similar to the original image in the feature space of a discriminative network.”
Towards a Neural Statistician (2016, methodology): “Our architecture for this problem is based on one presented in [19].”
Towards Deeper Understanding of Variational Autoencoding Models (2017, neutral): “Correspondence to: Shengjia Zhao < zhaosj12@stanford.edu > , Jiaming Song < tsong@stanford.edu > , Stefano Ermon < er-mon@stanford.edu > . et al., 2015; Lamb et al., 2016).”
Super-Resolution with Deep Convolutional Sufficient Statistics (2015, methodology): “The recent work by Lamb et al. (2016) proposed to use representations derived from CNNs trained on discriminative tasks to regularize the objective function of the variational autoencoder.”
Learning a Driving Simulator (2016, background): “[24] proposed discriminative generative networks that uses distances between features of a previously trained classiﬁer as part of the cost function.”
Hierarchical Adversarially Learned Inference (2018, methodology): “Examples of this include Larsen et al. (2015) that combines the GAN generator and the VAE decoder into one network and Lamb et al. (2016) that uses the loss of a pre-trained classiﬁer as an additional reconstruction loss in the VAE objective.”
Transformation-Grounded Image Generation Network for Novel 3D View Synthesis (2017, background): “L 1 norm [36], 4th-feature reconstruction loss with pretrained VGG16 network [19, 25, 37, 24], 5th-adversarial loss with feature matching [13, 32, 34], 6th-the combined loss).”
Improving Consistency and Correctness of Sequence Inpainting using Semantically Guided Generative Adversarial Network (2017, methodology): “Conditional inputs have also been used as discriminative regularizers [24] for improving VAE sample quality.”
Visual Robot Task Planning (2018, background): “In fact, previous work suggests that incorporating a discriminator into a generative model will increase crispness and quality of results [26].”
Generative Compression (2017, background): “Although GANs provide an appealing method for reconstructing quality images from their latent code, they lack the inference (encoder) function f : X → Z necessary for image compression.”
Deep Feature Consistent Variational Autoencoder (2016, background): “[15] proposes to enhence the objective function with discriminative regularization.”
Drawing and Recognizing Chinese Characters with Recurrent Neural Network (2016, background): “ularization [58] to improve the quality of the generative model.”