Mixtures of Conditional Gaussian Scale Mixtures Applied to Multiscale Image Representations

By Lucas Theis, Reshad Hosseini, Matthias Bethge. 2011

39 citations recorded by the source corpus.

We present a probabilistic model for natural images that is based on mixtures of Gaussian scale mixtures and a simple multiscale representation. We show that it is able to generate images with interesting higher-order correlations when trained on natural images or samples from an occlusion-based model. More importantly, our multiscale model allows for a principled evaluation. While it is easy to generate visually appealing images, we demonstrate that our model also yields the best performance reported to date when evaluated with respect to the cross-entropy rate, a measure tightly linked to the average log-likelihood. The ability to quantitatively evaluate our model differentiates it from other multiscale models, for which evaluation of these kinds of measures is usually intractable.

Subfields: Biophysics, Cognitive Neuroscience, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

Learning to generate chairs with convolutional neural networks (2014, neutral): “This includes a wide variety of methods ranging from Gaussian mixture models [9, 31] to autoregressive models [19] and stochastic variations of neural networks [2, 11, 25, 17, 30].”
PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications (2017, positive): “Instead of directly modeling the discrete pixel values in an image, it is also possible to de-quantize them by adding noise from the standard uniform distribution, as used by Uria et al. (2013) and others, and modeling the data as being continuous.”
Deep Unsupervised Learning using Nonequilibrium Thermodynamics (2015, neutral): “obabilistic model trained on images of bark. Note the long-range spatial structure, for instance in the crack entering on the left side of the inpainted region. Model Log Likelihood Dead Leaves MCGSM [42] 1.244 bits/pixel Diffusion 1:489 bits/pixel MNIST Stacked CAE [2] 121 passed through a sigmoid ”
RNADE: The real-valued neural autoregressive density-estimator (2013, neutral): “For image patches, a scale mixture can work well [11], and could be explored within our framework.”
Amortised MAP Inference for Image Super-resolution (2016, positive): “, 2016) but with a continuous (and differentiable) MCGSM (Theis et al., 2012) likelihood.”
How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary? (2015, neutral): “In this section we will focus on a particularly useful class of probabilistic models, which we call autoregressive generative models (see e. g. Theis et al., 2012; Larochelle & Murray, 2011; Bengio et al., 2015).”
Video (language) modeling: a baseline for generative models of natural videos (2014, neutral): “While several authors have reported realistic generation of small image patches, fewer works have operated at the scale of high-resolution images (Theis et al., 2012; Ranzato et al., 2013) and success has been more limited.”
Density Modeling of Images using a Generalized Normalization Transformation (2015, neutral): “…this induced density model more thoroughly to other model forms that have been proposed in the literature (e.g., ﬁnite mixtures of Gaussians or GSMs (Guerrero-Colón et al., 2008; Lyu & Simoncelli, 2009a; Zoran & Weiss, 2012; Theis et al., 2012), and sparse factorization (Culpepper et al., 2011)).”
Learning to Generate Chairs, Tables and Cars with Convolutional Networks (2014, unknown): “This includes a wide variety of methods ranging from Gaussian mixture models [10], [11] to autoregressive models [12] and stochastic variations of neural networks [13], [14], [15], [16], [17].”
Diverse feature visualizations reveal invariances in early layers of deep neural networks (2018, unknown): “In a nutshell, PixelCNN++ improves upon PixelCNN [23] and earlier autoregressive models [24, 30, 31] that attempt to capture the distribution of natural images by expressing the joint distribution of all pixels as the product of the distributions of individual pixels conditioned on a causal neighbor”
The student-t mixture as a natural image patch prior with application to image compression (2014, unknown): “Another model related to the GMM and STM is the Mixture of Gaussian scale mixtures (MoGSM) (Theis et al., 2011, 2012).”
CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training (2017, unknown): “Conventional wisdom and early research of generative models, including Principle Component Analysis (PCA) [40], Independent Component Analysis (ICA) [10],and the Gaussian Mixture Model (GMM) [46, 27, 37], all assume a simple formation of data.”
Fine-Grained Similarity Measurement between Educational Videos and Exercises (2020, unknown): “[31] used the conditional distribution of a mixture of GSMs (gaussian scale mixtures) [34, 37] to model the distribution of a pixel given its causal neighborhood.”