A Practical Guide to Training Restricted Boltzmann Machines

By Geoffrey E. Hinton. 2012

3,201 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Probabilistic & Bayesian Methods

Citations in the corpus

Generating Sequences With Recurrent Neural Networks (2013, background): “Note that the densities are normalised (up to a fixed constant) and are therefore straightforward to differentiate and pick unbiased sample from, in contrast with restricted Boltzmann machines [14] and other undirected models.”
Residual Energy-Based Models for Text Generation (2020, methodology): “When the partition function is intractable, Maximum Likelihood Estimation (MLE) requires samples from the model distribution, which is usually approximated with Monte Carlo sampling or mean ﬁeld inference (Hinton, 2012; LeCun et al., 2006) for globally normalized models.”
On the Anatomy of MCMC-based Maximum Likelihood Learning of Energy-Based Models (2019, neutral): “Early energy-based models such as the RBM and FRAME model use Gibbs sampling as the MCMC method.”
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models (2020, methodology): “Early variants of EBMs include models whose energy is defined over both data and auxiliary latent variables (Salakhutdinov & Hinton, 2009; Hinton, 2012), and models using only data variables (Hinton, 2002; Mnih & Hinton, 2005).”
Representation Learning: A Review and New Perspectives (2012, methodology): “(Ranzato and Hinton, 2010) introduced the mean and covariance RBM (mcRBM).”
What do we need to build explainable AI systems for the medical domain? (2017, methodology): “The probabilistic semantics for an RBM is deﬁned by its energy function (for details see the chapter by Hinton (2012).”
Methods for interpreting and understanding deep neural networks (2017, methodology): “are often used to make most efficient use of these methods and extend their capabilities [10, 25, 47].”
This looks like that: deep learning for interpretable image recognition (2018, methodology): “Examples of posthoc analysis techniques include activation maximization [5, 12, 22, 42, 30, 36, 47], deconvolution [48], and saliency visualization [36, 40, 39, 34].”
Deep Gaussian Processes (2012, methodology): “d it is useful to have an objective measure of the model’s applicability. The traditional approach to deep learning is based around binary latent variables and the restricted Boltzmann machine (RBM) [Hinton, 2010]. Deep hierarchies are constructed by stacking these models and various approximate inf”
Which Artificial Intelligence Algorithm Better Predicts the Chinese Stock Market? (2018, background): “The probability distributions over hidden or visible vectors in general Boltzmann machines are deﬁned in terms of the energy function [31]: where Z is a partition function deﬁned as the sum of e − G ( X , Y ) over all possible conﬁgurations, and the marginal probability of a visible vector is the…”
Deep Learning: Methods and Applications (2014, methodology): “See Technical Report [159] for a very useful practical guide for training RBMs.”
Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data (2016, methodology): “Except for the two OCD GR speciﬁc parameters, the settings for the others are discussed by Hinton (2012).”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, background): “The panorama presented in this chapter regards some of these surviving or novel elements of practice, focusing on learning algorithms aiming at training deep neural networks, but leaving most of the material specific to the Boltzmann machine family to another chapter (Hinton, 2013).”
Deep Directed Generative Models with Energy-Based Probability Estimation (2016, background): “r, and the responses of all weak classiﬁer are aggregated to assign an energy value: P~ i (x) = ˙ WT i x+ b i E (x) = X i log 1 + e (W i Tx+b) (4) Interestingly, restricted Boltzmann machines (RBM) [8] have a free energy over Gaussian visible units x with terms associated with binary hidden units h”
Learning Where to Attend with Deep Architectures for Image Tracking (2011, methodology): “These features can be computed from static images (Torralba, Oliva, Castelhano, & Henderson, 2006; Barrington, Marks, Hsiao, & Cottrell, 2008) or from local regions of space time (Gaborski, Vaingankar, Chaoji, Teredesai, & Tentler, 2004) for video. Additionally, a wide variety of different feature t”
PacGAN: The Power of Two Samples in Generative Adversarial Networks (2017, background): “Deep neural network-based generative models were proposed to cope with this problem [22, 15, 14].”