Factoring Variations in Natural Images with Deep Gaussian Mixture Models

By Aäron van den Oord, Benjamin Schrauwen. 2014

100 citations recorded by the source corpus.

Subfields: Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling (2019, neutral): “48 D EEP GMM S [58] = 4 .”
Improved Variational Inference with Inverse Autoregressive Flow (2016, background): “(van den Oord et al., 2016) 4.70 NICE (Dinh et al., 2014) 4.48 Deep GMMs (van den Oord and Schrauwen, 2014) 4.00 Real NVP (Dinh et al., 2016) 3.49 Pixel CNN van den Oord et al. (2016) 3.14 Pixel RNN (Row LSTM) van den Oord et al. (2016) 3.07 Pixel RNN (Diagonal BiLSTM) van den Oord et al. (2016)…”
A note on the evaluation of generative models (2015, methodology): “To avoid this case, it is becoming best practice to add real-valued noise to the integer pixel values to dequantize the data (e.g., Uria et al., 2013; van den Oord & Schrauwen, 2014; Theis & Bethge, 2015).”
Generative Image Modeling Using Spatial LSTMs (2015, methodology): “Generative image models are also useful in more traditional applications such as image reconstruction [33, 35, 49] or compression [47].”
Continual Learning with Fully Probabilistic Models (2021, methodology): “A solution to both problems could be the use of deep GMM variants as presented in [39] or [33].”
Locally-connected transformations for deep GMMs (2015, methodology): “In (van den Oord & Schrauwen, 2014) all log-likelihood evaluations were done exactly, by summing over all possible paths.”
Approximate Inference for Deep Latent Gaussian Mixtures (2016, methodology): “Our deep Latent Gaussian mixture model (DLGMM) generalizes previous work such as Factor Mixture Analysis [12] and Deep Gaussian Mixtures [15] to arbitrary differentiable inter-layer transformations.”
Image Modeling with Deep Convolutional Gaussian Mixture Models (2021, methodology): “This is an argument similar to the one about different paths through a hierarchical MFA model in [31], [32].”
Practical Gauss-Newton Optimisation for Deep Learning (2017, methodology): “Our choice was motivated by recent interest in neural network mixture models (Eigen et al., 2013; Zen & Senior, 2014; van den Oord & Schrauwen, 2014; Shazeer et al., 2017); our mixture model is also appropriate for testing the performance of KFLR. Training was run for 40 , 000 updates for ADAM with…”
A RAD approach to deep mixture models (2019, background): “While this is not necessary, we will exploit this structure as highlighted later in the main text and in Figure 4. et al., 2017), some of which can be expressed as a deep mixture model (Tang et al., 2012; Van den Oord & Schrauwen, 2014; van den Oord & Dambre, 2015).”
Relaxing Bijectivity Constraints with Continuously Indexed Normalising Flows (2019, methodology): “A special case of this framework is given by deep Gaussian mixture models (Van den Oord & Schrauwen, 2014; van den Oord & Dambre, 2015), which corresponds to using invertible linear transformations for each fi.”
A Provably Correct Algorithm for Deep Learning that Actually Works (2018, result): “Finally, the works of [13, 14, 16] study generative models similar to ours, with promising empirical results when implementing EM inspired algorithms, but giving no theoretical foundations whatsoever.”