Variational MCMC

By Nando de Freitas, Pedro A. d. F. R. Højen-Sørensen, Stuart J. Russell. 2001

107 citations recorded by the source corpus.

We propose a new class of learning algorithms that combines variational approximation and Markov chain Monte Carlo (MCMC) simulation. Naive algorithms that use the variational approximation as proposal distribution can perform poorly because this approximation tends to underestimate the true variance and other features of the data. We solve this problem by introducing more sophisticated MCMC algorithms. One of these algorithms is a mixture of two MCMC kernels: a random walk Metropolis kernel and a block Metropolis-Hastings (MH) kernel with a variational approximation as proposal distribution. The MH kernel allows one to locate regions of high probability efficiently. The Metropolis kernel allows us to explore the vicinity of these regions. This algorithm outperforms variational approximations because it yields slightly better estimates of the mean and considerably better estimates of higher moments, such as covariances. It also outperforms standard MCMC algorithms because it locates the regions of high probability quickly, thus speeding up convergence. We also present an adaptive MCMC algorithm that iterates between improving the variational approximation and improving the MCMC approximation. We demonstrate the algorithms on the problem of Bayesian parameter estimation for logistic (sigmoid) belief networks.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Probabilistic & Bayesian Methods

Citations in the corpus

A-NICE-MC: Adversarial Training for MCMC (2017, methodology): “Unlike variational approaches, MCMC methods are iterative in nature and do not naturally lend themselves to the use of expressive function approximators [6, 7].”
Towards Biologically Plausible Deep Learning (2015, methodology): “Variational MCMC (de Freitas et al., 2001) can be used to approximate the posterior, e.g., as in the model from Salimans et al. (2014). However, a rejection step does not look very biologically plausible (both for the need of returning to a previous state and for the need to evaluate the joint likel”
Learning Deep Latent Gaussian Models with Markov Chain Monte Carlo (2017, methodology): “The idea of using MCMC to improve variational approximations has come up a number of times (e.g., De Freitas et al., 2001; Mimno et al., 2012).”
Regression Density Estimation With Variational Methods and Stochastic Approximation (2012, methodology): “MCMC methods and variational methods can be complementary, in the sense that variational methods are able to provide good proposal distributions for MCMC schemes, a strategy that is sometimes called variational MCMC (de Freitas et al. 2001).”
Diffusion of Lexical Change in Social Media (2012, result): “Similar hybrid approaches have been proposed in the prior literature [32].”
Neural Network Renormalization Group (2018, methodology): “One may thus employ the NeuralRG network as a trainable MCMC proposal [51, 60].”
An Introduction to MCMC for Machine Learning (2004, background): “Recently, it has been argued that the combination of MCMC and variational optimisation techniques can also lead to more efficient sampling (de Freitas et al., 2001).”