A Better Way to Pretrain Deep Boltzmann Machines

By Ruslan Salakhutdinov, Geoffrey E. Hinton. 2012

136 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Yoshua Bengio
2. Kyunghyun Cho
3. Dacheng Tao
4. Yi Yang
5. Ruslan Salakhutdinov
6. Liang Zheng
7. Quoc V. Le
8. Qi Tian

Citations in the corpus

• Leveraging Meta-path based Context for Top- N Recommendation with A Neural Co-Attention Model (2018, methodology): “Inspired by the tricks for training neural networks [11, 12], we propose to use a similar pretrain technique to measure the priority degree of each candidate out-going nodes.”
• Modeling Documents with Deep Boltzmann Machines (2013, methodology): “In this DBM, the second-layer performs 1 / 2 of the modeling work compared to the first layer [11].”
• Multimodal Large Language Models: A Survey (2023, methodology): “In 2012, a multimodal learning algorithm based on deep Boltzmann machines (DBMs) [20] aimed to model the dependencies and interactions between different modalities.”
• Learning Affective Features With a Hybrid Deep Model for Audio–Visual Emotion Recognition (2018, background): “Due to such good property, DBNs and its variant called Deep Bolzmann Machines (DBMs) [25] have been successfully utilized to learn high-level feature representations from low-level hand-crafted features for multimodal emotion recognition [26]–[28].”
• A Two-Stage Pretraining Algorithm for Deep Boltzmann Machines (2013, methodology): “When the conventional pretraining algorithm is used, we do not explicitly make duplicate copies of visible or hidden units, but only double the corresponding weight parameters [34].”
• Deep learning for visual understanding: A review (2016, background): “The improvements can either take place at the pre-training stage [68,69] or at the training stage [70,71].”
• Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN (2020, methodology): “data-free demand. GAN [9] has shown the capability of generating better highquality images [28,14], compared to existing methods such as Restricted Boltzmann Machines [12] and Deep Boltzmann Machines [23]. A vanilla GAN model [9] has two important components, i.e. a generator and a discriminator. Th”
• Deep learning for visual understanding (2017, unknown): “The improvements can either take place at the pre-training stage [94, 95] or at the training stage [96, 97].”
• Neural Networks and Deep Learning (2018, unknown): “Deep Boltzmann machines are discussed in [417], and efficient algorithms are discussed in [200, 418].”