Approximate Inference for Deep Latent Gaussian Mixtures

By Eric T. Nalisnick, Lars Hertel, Padhraic Smyth. 2016

100 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Xu Tan
2. Yu Zhang
3. Yoshua Bengio
4. Tao Qin
5. Wei-Ning Hsu
6. Ye Jia
7. Quoc V. Le
8. Sergey Levine

Citations in the corpus

• Continual Unsupervised Representation Learning (2019, methodology): “Building on the original VAE (Kingma & Welling, 2013), Nalisnick et al. (2016) utilise a latent mixture of Gaussians, aiming to capture class structure in an unsupervised fashion, and propose a Bayesian non-parametric prior, further developed in (Nalisnick & Smyth, 2017).”
• Hierarchical Generative Modeling for Controllable Speech Synthesis (2018, methodology): “Using a mixture distribution for latent variables in a VAE was explored in Dilokthanakul et al. (2016); Nalisnick et al. (2016), and Jiang et al. (2017) for unconditional image generation and text topic modeling.”
• VAE with a VampPrior (2017, methodology): “These effects distinguish the VampPrior from the MoG prior utilized in the VAE so far [9, 29].”
• Sylvester Normalizing Flows for Variational Inference (2018, background): “Therefore, designing tractable and more expressive variational families is an important problem in variational inference (Nalisnick et al., 2016; Salimans et al., 2015; Tran et al., 2015).”
• Taming VAEs (2018, background): “As a result, several mechanisms have been proposed to increase the expressiveness of the variational posterior density [21, 22, 23, 24, 25, 3, 26, 5] but it still remains a challenge to train complex encoders due to a combination of optimization and generalization issues [27, 28].”
• Resampled Priors for Variational Autoencoders (2018, methodology): “A mixture of Gaussian prior (Nalisnick et al., 2016; Dilokthanakul et al., 2016) provides a simpler alternative to the VampPrior, but is harder to optimize and does not perform as well (Tomczak and Welling, 2018).”
• Variational Memory Addressing in Generative Models (2017, neutral): “A number of architectures for VAEs augmented with mixture priors have been proposed, but they do not use the mixture component indicator variable to index memory and integrate out the variable instead [7, 22], which prevents them from scaling to a large number of mixing components.”
• Dirichlet Variational Autoencoder (2019, background): “MNIST (K = 50) OMNIGLOT (K = 100) k = 3 k = 5 k = 10 k = 3 k = 5 k = 10 GVAE (Nalisnick et al., 2016) 28.”
• Image-to-Image Translation: Methods and Applications (2021, background): “The studies in [59], [60], [61] enrich the variational posterior to alleviate the blurriness of generated samples.”
• Generalized ELBO with Constrained Optimization , GECO (2018, background): “As a result, several mechanisms have been proposed to increase the expressiveness of the variational posterior density [6, 7, 8, 9, 10, 11, 12, 13, 14] but it still remains a challenge to train complex encoders due to a combination of optimization and generalization issues [15, 16].”
• Deep Bayesian Unsupervised Lifelong Learning (2021, background): “In (c), we assume that the generative model pθ(x|z) is parameterized by gθ : Z → X and gθ(z) = ( μ(z; θ),σ(2)(z; θ) ) , where gθ is chosen as DNNs due to its powerful function approximation and good feature learning capabilities [29, 30, 31].”