Learning in Markov Random Fields using Tempered Transitions

By Ruslan Salakhutdinov. 2009

120 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Better Mixing via Deep Representations (2012, methodology): “To improve mixing, a powerful idea that has been explored recently for deep learning algorithms (Desjardins et al. , 2010; Cho et al. , 2010; Salakhutdinov, 2010b,a) is tempering (Neal, 1994).”
Neural Autoregressive Distribution Estimation (2016, methodology): “Another line of development has been to optimize likelihood using RobbinsMonro stochastic approximation (Younes, 1989), also known as Persistent CD (Tieleman, 2008), and develop good MCMC samplers for deep undirected models (Salakhutdinov, 2009, 2010; Desjardins et al., 2010; Cho et al., 2010).”
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model (2019, methodology): “is Markov-chain Monte-Carlo (MCMC) sampling (Neal, 1993; Swendsen and Wang, 1986; Salakhutdinov, 2009; Desjardins et al., 2010; Cho et al., 2010).”
Parallel Tempering for Training of Restricted Boltzmann Machines (2010, methodology): “In work which was done concurrently but independently of ours, (Salakhutdinov, 2010) uses the method of tempered transitions (Neal, 1994) to promote mixing and fine-tuning the parameters of Markov random fields.”
Deep Logic Networks: Inserting and Extracting Knowledge From Deep Belief Networks (2018, unknown): “Deep architectures [3], [20], [23]–[25], on the other hand, are connectionist systems that consist of many hidden layers and therefore are capable of learning representations at multiple levels of abstraction.”
Foundations and Advances in Deep Learning (2014, methodology): “This algorithm, sometimes called stochastic approximation procedure (Salakhutdinov, 2009), does not run the Gibbs sampling chain, starting from random states until the convergence at each update.”
Training restricted Boltzmann machines: An introduction (2014, background): “3 Parallel tempering One of the most promising sampling techniques used for RBM training so far is parallel tempering (PT) [44, 12, 10].”
On Tracking The Partition Function (2011, methodology): “Other algorithms improve upon this default formulation by replacing Gibbs sampling with more powerful sampling algorithms [26, 7, 21, 20].”
The flip-the-state transition operator for restricted Boltzmann machines (2013, unknown): “…Technology HIIT, P.O. Box 68, 00014 Helsinki, Finland A. Fischer Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany e-mail: asja.fischer@ini.rub.de C. Igel ( ) Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark e-mail: igel@diku.dk”
An Efficient Learning Procedure for Deep Boltzmann Machines (2012, methodology): “As an alternative, we could adopt a variation of the Chib-style estimator, proposed by Murray and Salakhutdinov (2009).”
Enhanced Gradient for Training Restricted Boltzmann Machines (2013, methodology): “…deep belief network and the deep Boltzmann machine have been applied to various machine learning tasks with impressive improvements over conventional approaches (Hinton & Salakhutdinov, 2006; Salakhutdinov & Hinton, 2009; Salakhutdinov, 2009; Krizhevsky, 2010; Lee, Grosse, Ranganath, & Ng, 2009).”
Training Restricted Boltzmann Machines (2015, methodology): “Parallel Tempering (PT, also known as replica exchange Monte Carlo sampling) has also been applied to RBMs (Cho et al., 2010; Desjardins et al., 2010b; Salakhutdinov, 2009).”
Enhanced Gradient and Adaptive Learning Rate for Training Restricted Boltzmann Machines (2011, methodology): “Annealed importance sampling (AIS) provides a way to estimate the normalizing constant of RBM (Salakhutdinov, 2009a).”
Learning Deep Boltzmann Machines using Adaptive MCMC (2010, methodology): “When learning Restricted Boltzmann Machines (RBM’s), (Desjardins et al., 2010; Salakhutdinov, 2010) show that this can often lead to poor generative models.”