Joint Training Deep Boltzmann Machines for Classification

By Ian J. Goodfellow, Aaron C. Courville, Yoshua Bengio. 2013

25 citations recorded by the source corpus.

We introduce a new method for training deep Boltzmann machines jointly. Prior methods of training DBMs require an initial learning pass that trains the model greedily, one layer at a time, or do not perform well on classification tasks. In our approach, we train all layers of the DBM simultaneously, using a novel training procedure called multi-prediction training. The resulting model can either be interpreted as a single generative model trained to maximize a variational approximation to the generalized pseudolikelihood, or as a family of recurrent networks that share parameters and may be approximately averaged together using a novel technique we call the multi-inference trick. We show that our approach performs competitively for classification and outperforms previous methods in terms of accuracy of approximate inference and classification with missing inputs.

Subfields: Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

In Signal Processing

Citations in the corpus

Deep Learning of Representations: Looking Forward (2013, methodology): “Another key ingredient in the success of many recent breakthrough results in the area of object recognition is the idea of dropouts (Hinton et al., 2012b; Krizhevsky et al., 2012; Goodfellow et al., 2013b).”
Neuronal Synchrony in Complex-Valued Deep Networks (2013, methodology): “…as a probabilistic model, but for our purpose here it is more appropriate to simply think of a DBM as a multi-layer recurrent neural network (cf. Goodfellow et al., 2013a) with logistic activation function;5 we can demonstrate how our framework works by taking the pretrained network,…”
Deep learning for visual understanding: A review (2016, methodology): “The improvements can either take place at the pre-training stage [68,69] or at the training stage [70,71].”
General Stochastic Networks for Classification (2014, unknown): “94% MP-DBM [28] 0.”
Deep learning for visual understanding (2017, unknown): “The improvements can either take place at the pre-training stage [94, 95] or at the training stage [96, 97].”