An Introduction to Variational Methods for Graphical Models

By Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola, Lawrence K. Saul. 1999

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Energy-Inspired Models: Learning with Sampler-Induced Distributions (2019, background): “This allows us to maximize variational lower bounds [32, 7] on the log-likelihood (c.f., Kingma and Welling [37], Rezende et al. [59]).”
BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modeling (2019, methodology): “In order to circumvent this, we use Variational Inference [19] and introduce a posterior approximation q φ ( z | x ) , known as inference network or encoder , that is parameterized by a neural network φ .”
Variational Inference with Normalizing Flows (2015, methodology): “Another potentially powerful alternative would be to specify the approximate posterior as a mixture model, such as those developed by Jaakkola & Jordan (1998); Jordan et al. (1999); Gershman et al. (2012).”
Deep Unsupervised Learning using Nonequilibrium Thermodynamics (2015, background): “…exist which ameliorate, but do not remove, this tradeoff – for instance mean field theory and its expansions [39, 40], variational Bayes [19], contrastive divergence [44, 13], minimum probability flow [36, 35], minimum KL contraction [28], proper scoring rules [9, 32], score matching…”
Foundation Posteriors for Approximate Probabilistic Inference (2022, neutral): “Markov Chain Monte Carlo (MCMC) [32, 24] and variational inference (VI) [37, 64, 8] are two widely used examples.”
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images (2015, methodology): “[15] for an overview) we resort to sampling zt from an approximate posterior distribution Qφ(Z|X) with parameters φ.”
Semantic Image Segmentation via Deep Parsing Network (2015, methodology): “Q(y) is typically optimized by minimizing a free energy function [15] of MRF,”
Stochastic variational inference (2012, methodology): “In variational inference, we deﬁne a ﬂexible family of distributions over the hidden variables, indexed by free parameters (Jordan et al., 1999; Wainwright and Jordan, 2008).”
Variational Bayesian Inference with Stochastic Search (2012, methodology): “Mean-ﬁeld variational Bayesian (MFVB) inference is an optimization-based approach to approximating the full posterior of the latent variables of a Bayesian model (Jordan et al., 1999).”
One-Shot Generalization in Deep Generative Models (2016, methodology): “One popular approximation technique is based on variational inference (Jordan et al., 1999), which transforms the difficult integration into an optimization problem that is typically more scalable and easier to solve.”
Collaborative Deep Learning for Recommender Systems (2014, methodology): “Based on the CDL model above, all parameters could be treated as random variables so that fully Bayesian methods such as Markov chain Monte Carlo (MCMC) or variational approximation methods [14] may be applied.”
Variational Autoencoders for Collaborative Filtering (2018, neutral): “We resort to variational inference [22].”
Recurrent Latent Variable Networks for Session-Based Recommendation (2017, methodology): “Variational inference for the developed ReLaVaR model consists in maximization of a lower-bound to the log-marginal likelihood of the model (evidence lower-bound, ELBO) w.r.t. the model parameters [16].”
Variational Inference: A Review for Statisticians (2016, methodology): “In the original literature on variational inference, this was derived through Jensen’s inequality (Jordan et al. 1999).”
Neural Variational Inference for Text Processing (2015, background): “Markov chain Monte Carlo (MCMC) (Neal, 1993; Andrieu et al., 2003) and variational inference (Jordan et al., 1999; Attias, 2000; Beal, 2003) are the standard approaches for approximating these integrals.”