Deep Boltzmann Machines

By Ruslan Salakhutdinov, Geoffrey E. Hinton. 2009

2,353 citations recorded by the source corpus.

Subfields: Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

Generative Adversarial Nets (2014, background): “In this family of model, perhaps the most succesful is the deep Boltzmann machine [25].”
Universal Language Model Fine-tuning for Text Classification (2018, methodology): “3 An unrelated method of the same name exists for deep Boltzmann machines (Salakhutdinov and Hinton, 2009).”
Pylearn2: a machine learning research library (2013, methodology): “Additionally, there are many kinds of preprocessing, such as PCA [38], ZCA [4], various kinds of local contrast normalization [46], as well as helper functions to set up the entire preprocessing pipeline from some well-known successful and documented systems [8, 45].”
Better Mixing via Deep Representations (2012, neutral): “…classiﬁcation but also better quality samples (in the sense of better corresponding to the target distribution being learned) is not novel and several observations support this hypothesis already, some quantitatively (Salakhutdinov and Hinton, 2009), some more qualitative (Hinton et al. , 2006).”
What makes ImageNet good for transfer learning? (2016, background): “The most well-known such criteria are image reconstruction [7, 38, 31, 29, 34, 23] (see [6] for a comprehensive overview) and feature slowness [48, 18].”
Fine-tuned Language Models for Text Classification (2018, methodology): “A method of the same name exists for deep Boltzmann machines (Salakhutdinov and Hinton, 2009).”
Neural Autoregressive Distribution Estimation (2016, background): “vely for developing powerful generative networks. These include the restricted Boltzmann machine (Smolensky, 1986; Freund and Haussler, 1992) and its multilayer extension, the deep Boltzmann machine (Salakhutdinov and Hinton, 2009), which dominate the literature on undirected neural networks. Salakh”
Jukebox: A Generative Model for Music (2020, methodology): “Modeling the interdependency within high-dimensional data was traditionally considered extremely difﬁcult, but starting with Deep Boltzmann Machines (Salakhutdinov & Hinton, 2009), various kinds of deep generative models have been introduced.”
Contrastive Multiview Coding (2019, methodology): “ve of downstream tasks such as object recognition. Representations better suited to such tasks have been learnt using deep neural networks, starting with seminal techniques such as Boltzmann machines [55, 50], autoencoders [23], variational autoencoders [30], generative adversarial networks [20] and”
Neural Discrete Representation Learning (2017, background): “…with continuous features have been the focus of many previous work [16, 39, 6, 9] however we concentrate on discrete representations [27, 33, 8, 28] which are potentially a more natural ﬁt for many of the modalities we are interested in. Language is inherently discrete, similarly speech…”
Unsupervised Visual Representation Learning by Context Prediction (2015, background): “To deal with these computational issues, a number of works, such as the wake-sleep algorithm [23], contrastive divergence [22], deep Boltzmann machines [45], and variational Bayesian methods [28, 43] use sampling to perform approximate inference.”
Learning to generate chairs with convolutional neural networks (2014, background): “Prominent examples from this line of work are restricted Boltzmann machines (RBMs) [13] and Deep Boltz-mann Machines (DBMs) [26], as well as the plethora of models derived from them [12, 22, 20, 29, 23].”
Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling (2020, background): “Most work focuses on the maximum likelihood learning of energy-based models [34–36].”
Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model (2019, methodology): “The maximum likelihood learning of the energy-based model (EBM) [29, 51, 19, 40, 30, 34, 6, 32, 48, 49, 22, 7, 47] follows what Grenander [15] called “analysis by synthesis” scheme.”
Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net (2017, background): “…et al., 2014)), or (2) energy function based undirected graphical models, corresponding to sampling from a stochastic process whose equilibrium stationary distribution obeys detailed balance with respect to the energy function (e.g. various Boltzmann machines (Salakhutdinov and Hinton, 2009)).”