A New Learning Algorithm for Mean Field Boltzmann Machines

By Max Welling, Geoffrey E. Hinton. 2002

155 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Probabilistic & Bayesian Methods

Citations in the corpus

Deep Unsupervised Learning using Nonequilibrium Thermodynamics (2015, background): “A variety of analytic approximations exist which ameliorate, but do not remove, this tradeoff – for instance mean field theory and its expansions [39, 40], variational Bayes [19], contrastive divergence [44, 13], minimum probability flow [36, 35], minimum KL contraction [28], proper scoring rules [9”
Minimum Probability Flow Learning (2009, background): “Contrastive divergence [24, 4] is a variation on steepest gradient descent of the maximum (log) likelihood (ML) objective function.”
Modeling image patches with a directed hierarchy of Markov random fields (2007, methodology): “Using ZCA has a similar effect to learning lateral connections between pixels (Welling and Hinton, 2002).”
Learning and Evaluating Boltzmann Machines (2008, methodology): “It was further observed [29, 18] that for Contrastive Divergence to perform well, it is important to obtain exact samples from the conditional distribution p(h|v).”
Deep Boltzmann Machines (2009, background): “It was further observed (Welling and Hinton, 2002; Hinton, 2002) that for Contrastive Divergence to perform well, it is important to obtain exact samples from the conditional distributionp(h|v; θ), which is intractable when learning full Boltzmann machines.”
Integrative Data Analysis of Multi-Platform Cancer Data with a Multimodal Deep Learning Approach (2015, methodology): “In practice, approximate learning approaches such as mean-field inference and Markov Chain Monte Carlo (MCMC), are often used to learn the parameters of an RBM model [12].”
Connectionist multivariate density-estimation and its application to speech synthesis (2016, methodology): “Larochelle and Murray developed NADE inspired by the mean-field variational approximation to the autoregressive conditionals of a binary RBM (Welling and Hinton, 2002)10.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, methodology): “Additionally, for contrastive divergence to perform well, exact samples from p(h|v; θ) are required, further making Boltzmann machines impractical [73], [213].”
Learning in Markov Random Fields with Contrastive Free Energies (2005, background): “Again, it is small and rarely in conflict with the other terms in the gradient so it can be safely ignored (see [17] for experimental evidence of this fact in the case of MF).”
Computational analysis and learning for a biologically motivated model of boundary detection (2008, methodology): “This means that solving the system of ODE's determining the activity of Stage II neurons amounts to performing the MFA on a Boltzmann Machine.”
Optimizing Neural Networks that Generate Iimages (2014, background): “Mean field CD (Welling & Hinton, 2002), abbreviated MF CD, is a variation on CD where we never make any stochastic choices, and just substitute (conditional) probabilities for the binary states that CD usually works with.”
Learning Algorithms for the Classification Restricted Boltzmann Machine (2012, unknown): “This approach was first described by Welling and Hinton (2002) and is known as mean field contrastive divergence.”