On distinguishability criteria for estimating generative models

By Ian J. Goodfellow. 2014

145 citations recorded by the source corpus.

Two recently introduced criteria for estimation of generative models are both based on a reduction to binary classification. Noise-contrastive estimation (NCE) is an estimation procedure in which a generative model is trained to be able to distinguish data samples from noise samples. Generative adversarial networks (GANs) are pairs of generator and discriminator networks, with the generator network learning to generate samples by attempting to fool the discriminator network into believing its samples are real data. Both estimation procedures use the same function to drive learning, which naturally raises questions about how they are related to each other, as well as whether this function is related to maximum likelihood estimation (MLE). NCE corresponds to training an internal data model belonging to the {\em discriminator} network but using a fixed generator network. We show that a variant of NCE, with a dynamic generator network, is equivalent to maximum likelihood estimation. Since pairing a learned discriminator with an appropriate dynamically selected generator recovers MLE, one might expect the reverse to hold for pairing a learned generator with a certain discriminator. However, we show that recovering MLE for a learned generator requires departing from the distinguishability game. Specifically: (i) The expected gradient of the NCE discriminator can be made to match the expected gradient of MLE, if one is allowed to use a non-stationary noise distribution for NCE, (ii) No choice of discriminator network can make the expected gradient for the GAN generator match that of MLE, and (iii) The existing theory does not guarantee that GANs will converge in the non-convex case. This suggests that the key next step in GAN research is to determine whether GANs converge, and if not, to modify their training algorithm to force convergence.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Evaluation & Benchmarks, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Evaluation & Benchmarks

In Signal Processing

Citations in the corpus

Improved Techniques for Training GANs (2016, methodology): “When used to seek for a Nash equilibrium, these algorithms may fail to converge [4].”
Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields (2017, background): “First (I), until recently it was not clear if in general gradient-based GAN learning could converge to one of the local Nash equilibria [38, 14, 13].”
Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis (2017, background): “Most recent work on GANs has focused on the task of recovering data distributions from natural images; many recent advances in image generation using GANs have shown great promise in producing photo-realistic natural images at high resolution [1–10] while attempting to solve many known problems with”
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization (2016, background): “[11], where an approximation to the gradient of the KL divergence is derived.”
NIPS 2016 Tutorial: Generative Adversarial Networks (2016, methodology): “Introduction This report1 summarizes the content of the NIPS 2016 tutorial on generative adversarial networks (GANs) (Goodfellow et al., 2014b).”
GANs Trained by a Two Time-Scale Update Rule Converge to a Nash Equilibrium (2017, background): “To characterize the convergence properties of training general GANs is still an open challenge [18, 19].”
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, background): “The Deep Learning community has recently proposed Generative Adversarial Networks (GANs) [29, 68, 70], which are still being intensively developed [3, 9, 31, 41, 54].”
Contrastive Representation Distillation (2019, background): “In (Goodfellow, 2014), it is shown that the NCE objective of Gutmann & Hyvärinen (2010) can lead to maximum likelihood learning, but not the adversarial objective.”
Censoring Representations with an Adversary (2015, methodology): “The loss we will use is of the form L = αC(X,R) + βD(S,R) + γE(Y,R) (4) where C is the cost of reconstructing X from R, D is a measure of dependence between R and S and E is the error in predicing Y from R.”
Connecting Generative Adversarial Networks and Actor-Critic Methods (2016, background): “It’s also worth noting that self-contrastive estimation [18] is conceptually similar to target networks, so the idea of a target network can still be applicable to other density estimation problems.”
Variational Approaches for Auto-Encoding Generative Adversarial Networks (2017, background): “This trick is widespread [11, 12, 17, 19, 26, 27, 39].”
Adversarial Ranking for Language Generation (2017, background): “However, as also discussed in the literature [8, 9], it is still an open problem how a non-Bernoulli GAN converges to such an equilibrium.”
Distribution Matching in Variational Inference (2018, methodology): “We will not have access to the marginal density, so instead we leverage the ability of classifiers to estimate density ratios, via the density ratio trick (Goodfellow et al., 2014; Goodfellow, 2014; Sugiyama et al., 2012).”
Improved generator objectives for GANs (2016, background): “[3] identiﬁes the generator objective that corresponds to minimizing the KL divergence, but does not empirically evaluate this objective.”
Deep and Hierarchical Implicit Models (2017, methodology): “Point estimates of the global parameters β sufﬁce for many applications (Goodfellow et al., 2014; Rezende et al., 2014).”
Maximum Entropy Generators for Energy-Based Models (2019, background): “2 and the objective function for Wasserstein GANs or WGAN (Ar-jovsky et al., 2017).”
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium (2017, background): “To characterize the convergence properties of training general GANs is still an open challenge [17, 18].”
Adversarial Contrastive Estimation (2018, background): “As noted in Goodfellow (2014), GAN learning does not implement maximum likelihood estimation (MLE), while NCE has MLE as an asymptotic limit.”
Skill Rating for Generative Models (2018, methodology): “…the two-player distinguishability game used by noise contrastive estimation (NCE) and generative adversarial networks (GANs) [Gutmann and Hyvarinen, 2010, Goodfellow et al., 2014, Goodfellow, 2014] and estimating the latent skill of generative models that participate in these tournaments.”