Efficient Learning of Deep Boltzmann Machines

By Ruslan Salakhutdinov, Hugo Larochelle. 2010

419 citations recorded by the source corpus.

Subfields: Generative Models, Representation & Self-Supervised Learning, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

1. Yoshua Bengio
2. Aaron C. Courville
3. Stefano Ermon
4. Max Welling
5. Diederik P. Kingma
6. Kyunghyun Cho
7. Jun Zhu
8. Ying Nian Wu

In Representation & Self-Supervised Learning

1. Yoshua Bengio
2. Ruslan Salakhutdinov
3. Aaron C. Courville
4. Kyunghyun Cho
5. Geoffrey E. Hinton
6. Pascal Vincent
7. Honglak Lee
8. Jasper Snoek

In Probabilistic & Bayesian Methods

1. Yoshua Bengio
2. Kyunghyun Cho
3. Aaron C. Courville
4. Ruslan Salakhutdinov
5. David M. Blei
6. Max Welling
7. Stefano Ermon
8. Hugo Larochelle

Citations in the corpus

• GAN Inversion: A Survey (2021, neutral): “(DBMs) [154], [155], [156], [157], Variational Autoencoders (VAEs) [158], [159], Deep Autoregressive Models (DARs) [160], [161], [162], [163], [164], and Normalizing Flow Models (NFMs) [165], [166], [167], [168].”
• On the challenges of learning with inference networks on sparse, high-dimensional data (2017, methodology): “One could alternatively, update φ using KL( ψ ( x ) M (cid:107) q φ ( z | x )) as in Salakhutdinov & Larochelle (2010). features, but in sparse datasets, many words appear only rarely.”
• Deep Learning of Representations: Looking Forward (2013, background): “See also Salakhutdinov and Larochelle (2010) for learned fast approximate variational inference in DBMs, or Bagnell and Bradley (2009); Stoyanov et al. (2011) for learning fast approximate inference (with fewer steps than would otherwise be required by standard general purpose inference) based on…”
• Energy-based Out-of-distribution Detection (2020, background): “Energy-based learning Energy-based machine learning models date back to Boltzmann machines [1, 35], networks of units with an energy defined for the overall network.”
• Inference Suboptimality in Variational Autoencoders (2018, background): “Salakhutdinov & Larochelle (2010) perform such local optimization when learning deep Boltzmann machines.”
• Deep learning for computational biology (2016, neutral): “…for modelling images (Box 2 ), recurrent neural networks for sequential data (Sutskever, 2013 ; Lipton, 2015 ), or restricted Boltzmann machines (Salakhutdinov & Larochelle, 2010 ; Hinton, 2012 ) and autoencoders (Hinton & Salakhutdinov, 2006 ; Alain et al , 2012 ; Kingma & Welling, 2013 ) for…”
• Neural Networks and Deep Learning (2018, neutral): “Deep Boltzmann machines are discussed in [417], and efficient algorithms are discussed in [200, 418].”
• An Introduction to Variational Autoencoders (2019, methodology): “In (Salakhutdinov and Larochelle, 2010) a recognition model was employed for efficient learning with Deep Boltzmann Machines.”
• Multiresolution Deep Belief Networks (2012, background): “Deep architectures such as the Deep Belief Network [1] (DBN), Deep Boltzmann Machine [2] (DBM) and Stacked Autoencoders [3] have been shown to be excellent at visual object recognition [4], speech phone recognition [5, 6], and image denoising [7].”
• A Two-Stage Pretraining Algorithm for Deep Boltzmann Machines (2013, methodology): “We used theMNIST and Caltech-101 Silhouettes datasets because experimental results of using DBMs for both datasets are readily available for direct comparison [36, 32, 27].”