On GANs and GMMs

By Eitan Richardson, Yair Weiss. 2018

165 citations recorded by the source corpus.

A longstanding problem in machine learning is to find unsupervised methods that can learn the statistical structure of high dimensional signals. In recent years, GANs have gained much attention as a possible solution to the problem, and in particular have shown the ability to generate remarkably realistic high resolution sampled images. At the same time, many authors have pointed out that GANs may fail to model the full distribution ("mode collapse") and that using the learned models for anything other than generating samples may be very difficult. In this paper, we examine the utility of GANs in learning statistical models of images by comparing them to perhaps the simplest statistical model, the Gaussian Mixture Model. First, we present a simple method to evaluate generative models based on relative proportions of samples that fall into predetermined bins. Unlike previous automatic methods for evaluating models, our method does not rely on an additional neural network nor does it require approximating intractable computations. Second, we compare the performance of GANs to GMMs trained on the same datasets. While GMMs have previously been shown to be successful in modeling small patches of images, we show how to train them on full sized images despite the high dimensionality. Our results show that GMMs can generate realistic samples (although less sharp than those of GANs) but also capture the full distribution, which GANs fail to do. Furthermore, GMMs allow efficient inference and explicit representation of the underlying statistical structure. Finally, we discuss how GMMs can be used to generate sharp images.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Generative Models, Probabilistic & Bayesian Methods, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Generative Models

In Probabilistic & Bayesian Methods

In Signal Processing

Citations in the corpus

DiffWave: A Versatile Diffusion Model for Audio Synthesis (2020, background): “The AM score and NDB are removed because they are less meaningful when the prior label distribution of X gen is speciﬁed.”
Continual Learning with Fully Probabilistic Models (2021, background): “With a reasonable memory overhead this may signiﬁcantly enhance the GMM’s sample generation capacity as demonstrated in, e.g., [40].”
Task Agnostic Continual Learning via Meta Learning (2019, methodology): “Since it is hard to visualize mode collapse for high dimensional image data, we use two bin-based metrics called NDB and JSD (Richardson & Weiss, 2018) to evaluate the resulting GANs.”
PacGAN: The Power of Two Samples in Generative Adversarial Networks (2017, background): “In a recent work in [57], it was empirically shown that Gaussian mixture models (GMM) can also generates realistic samples if trained eﬃciently, although the images are not as sharp as GAN generated samples.”
Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser (2020, background): “Although GANs can produce visually impressive synthetic samples, a number of results suggest that these samples are not representative of the full image density (a problem sometimes referred to as “mode collapse”) [12].”
ProDiff: Progressive Fast Diffusion Model for High-Quality Text-to-Speech (2022, methodology): “In addition, we employ two metrics NDB and JS [38] to explore the diversity of generated mel-spectrograms.”
Pros and Cons of GAN Evaluation Measures (2018, methodology): “To measure diversity of generates samples and mode collapse, Richardson and Weiss [43] propose an evaluation method based on the following observation: Given two sets of samples from the same distribution, the number of samples that fall into a given bin should be the same up to sampling noise.”
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations (2019, methodology): “To measure the similarity between the distribution between real images and generated one, we adopt two bin-based metrics, JSD and NDB (Richardson and Weiss 2018).”
PriorGAN: Real Data Prior for Generative Adversarial Nets (2020, methodology): “For example, Principle Component Analysis (PCA) [32], Independent Component Analysis (ICA) [13], and the Gaussian Mixture Model (GMM) [35, 26], all assume the data distribution follow a simple assumption.”
Image-Adaptive GAN based Reconstruction (2019, unknown): “Yet, as demonstrated in (Richardson and Weiss 2018), it is common to other recent learning approaches as well.”
A Non-Parametric Test to Detect Data-Copying in Generative Models (2020, methodology): “For completeness, we make use of a simple representation test that is essentially used by Richardson and Weiss (2018), now with an independent test set instead of the training set.”
A RAD approach to deep mixture models (2019, methodology): “A standard way to address this issue has been to use mixture models (Yeung et al., 2017; Richardson and Weiss, 2018; Eghbal-zadeh et al., 2018) or structured priors (Johnson et al.”
Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis (2019, methodology): “To measure the similarity between the distribution between real images and generated one, we adopt two bin-based metrics, NDB and JSD, proposed in [24].”
What makes fake images detectable? Understanding properties that generalize (2020, methodology): “Finally, we also include a face generator based on a Gaussian Mixture Model (GMM) rather than convolutional layers [31]; the GMM uses low-rank plus diagonal Gaussians to efficiently model covariance in high-dimensional outputs such as images.”
Image Modeling with Deep Convolutional Gaussian Mixture Models (2021, methodology): “The use of MFAs for describing natural images is discussed in detail in [28], showing that the MFA model alone, without further hierarchical structure, compares quite favorably to GANs when considering image generation.”