Do GANs learn the distribution? Some Theory and Empirics

By Sanjeev Arora, Andrej Risteski, Yi Zhang. 2018

171 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

Citations in the corpus

Are GANs Created Equal? A Large-Scale Study (2017, background): “As such, developing quantitative evaluation metrics is a critical research direction [3, 23].”
Imperfect ImaGANation: Implications of GANs Exacerbating Biases on Facial Data Augmentation and Snapchat Selfie Lenses (2020, background): “In reality, GAN-generated distributions are not nearly as diverse as their training distributions (Arora et al., 2018; 2017) and the support (i.e. possible feature combinations of the generated data) is only representative of a small subset of what one would expect to see when sampling data from…”
GAIT: A Geometric Approach to Information Theory (2019, neutral): “Our approach obtains similar results as (a form of) the birthday paradoxbased method of Arora et al. (2018), while avoiding the need for human evaluation of possible duplicates.”
Catastrophic forgetting and mode collapse in GANs (2020, background): “In this paper, we focus on the continual learning problem in the discriminator as many prior works have showed that the quality of a GAN mainly depends on its discriminator [16, 17, 18].”
Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting (2019, background): “In spite of the recent successes of deep generative models, existing theoretical results show that learning distributions in an unbiased manner is either impossible or has prohibitive sample complexity [4, 5].”
Detecting Overfitting of Deep Generative Networks via Latent Recovery (2019, methodology): “In [2], the support of a GAN generator, in terms of the number of face identities it could produce, was estimated using the birthday paradox heuristic.”
When can Wasserstein GANs minimize Wasserstein Distance? (2020, background): “In this case, the prior works [6, 4] have raised significant challenges to the central question: When the discriminator D is a neural network, it has been observed that when the D is too powerful, the generator G will suffer from mode collapse, i.”
On GANs and GMMs (2018, methodology): “Two examples are applying a perceptual multiscale similarity metric (MS-SSIM) on random patches [31] and, basing on the Birthday Paradox (BP), looking for the most similar pair of images in a batch [3].”
On Memorization in Probabilistic Deep Generative Models (2021, background): “We focus on explicit density models as these are more amenable to a direct analysis of the learned probability distribution (as opposed to implicit density models such as GANs).”
Hausdorff GAN: Improving GAN Generation Quality With Hausdorff Metric (2021, background): “That being said, these solutions may occur only from certain hyperparameters’ combinations [4] or from memorization of training samples [35].”
Prescribed Generative Adversarial Networks (2019, background): “When mode collapse happens, the generative distribution p θ ( x ) is degenerate and of low support (Arora et al., 2017, 2018).”
A Survey of Unsupervised Deep Domain Adaptation (2018, methodology): “es a few images with high probability (e.g. outputs one image 10 percent of the time), which would result in duplicate images being found in a set of 20 even though the support size may be very large [5]. However, the birthday paradox test still requires human interaction. To minimize human involvem”
Extracting Training Data from Diffusion Models (2023, background): “For instance, numerous metrics exist to measure similarity with the training data [32, 3], the extent of mode collapse [61, 15], and the impact of individual training samples [4, 75].”
On catastrophic forgetting and mode collapse in Generative Adversarial Networks (2018, background): “In this paper, we focus on the continual learning problem in the discriminator as many prior works have showed that the quality of a GAN mainly depends on its discriminator [16, 17, 18].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, background): “[306] proposed a novel test for estimating support size using the birthday paradox of discrete probability and show that GAN does suffer mode collapse even when images are of higher visual quality.”
Skill Rating for Generative Models (2018, background): “Some other metrics focus on the diversity of samples generated by the model [Arora et al., 2018, Santurkar et al., 2018].”
Making Method of Moments Great Again? -- How can GANs learn the target distribution (2020, background): “y does not have such an obvious answer in practice, when the set of functions Fis restricted to be certain class of neural networks instead of all 1-Lipschitz functions. In this case, the prior works [6, 4] have raised signicant challenges to the central question: When the discriminator Dis a neural”
Fair Generative Modeling via Weak Supervision (2019, background): “Even more, the sample complexity for learning high dimensional distributions can be doublyexponential in the dimensions in many cases [19], surpassing the size of the largest available datasets.”
Adversarial Transfer Learning (2018, methodology): “outputs one image 10 percent of the time), which would result in duplicate images being found in a set of 20 even though the support size may be very large [5].”