Classification using discriminative restricted Boltzmann machines

By Hugo Larochelle, Yoshua Bengio. 2008

823 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Statistical and Nonlinear Physics

Citations in the corpus

Deep Learning: Methods and Applications (2014, methodology): “In [203], the generative model of RBM is learned using the discriminative criterion of posterior class-label probabilities.”
Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data (2016, background): “Examples of these applications are: modeling human choice (Osogami and Otsuka, 2014), collaborative ﬁltering (Salakhutdinov et al, 2007), information retrieval (Gehler et al, 2006), transfer learning (Ammar et al, 2013), or multi-class classiﬁcation (Larochelle and Bengio, 2008).”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, background): “Another option is to penalize the biases of the hidden units, to make them more negative (Ranzato et al. , 2007; Lee et al. , 2008; Goodfellow et al. , 2009; Larochelle and Bengio, 2008).”
Training Deep Neural Networks on Noisy Labels with Bootstrapping (2014, background): “Larochelle & Bengio (2008) developed an RBM for classiﬁcation that uses a hybrid generative and discriminative training objective.”
Deep Learning of Representations: Looking Forward (2013, neutral): “Supervised and unsupervised objectives can of course be combined (simply added, with a hyper-parameter as coefficient), like in Larochelle and Bengio (2008)’s discriminative RBM.”
Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification (2016, methodology): “Several attempts (e.g., Ranzato & Szummer (2008); Larochelle & Bengio (2008); Sohn et al. (2013); Goodfel-low et al. (2013)) had been made to couple the unsupervised and supervised learning in the same phase, making unsupervised objectives able to impact the network training after supervised…”
Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science (2017, neutral): “RBMs, beside being very successful in providing very good initialization weights to the supervised training of deep artificial neural network architectures (Hinton et al, 2006), are also very successful as stand alone models in a variety of tasks, such as density estimation to model human choice (Os”
Bayesian Posterior Sampling via Stochastic Gradient Fisher Scoring (2012, positive): “We trained a DRBM (Larochelle & Bengio, 2008) on the KDD99 dataset which consists of 4,898,430 datapoints with 40 features, belonging to a total of 23 classes.”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, methodology): “This discriminative RBM can be used to “fine tune” RBM for classification tasks [142].”
The Neural Autoregressive Distribution Estimator (2011, methodology): “If the observations can be decomposed into an input x and a target y , then an RBM trained on such pairs can also be used to predict the missing target for new inputs (Larochelle & Bengio, 2008; Tieleman, 2008).”
Online Semi-Supervised Learning with Deep Hybrid Boltzmann Machines and Denoising Autoencoders (2015, background): “…ideas of (Lasserre et al., 2006), including the training schemes proposed in Ororbia II et al. (2015b,a), which built on the initial ideas of Larochelle & Bengio (2008); Larochelle et al. (2012), and of which the schemes of Calandra et al. (2012); Zhou et al. (2012) were shown to be special…”
A new deep convolutional neural network for fast hyperspectral image classification (2017, methodology): “1 A DBN is composed by a stack of restricted Boltzmann machines (Smolensky, 1986; Larochelle and Bengio, 2008) (RBMs).”
Learning to Disentangle Factors of Variation with Manifold Interaction (2014, positive): “…in the following fixed-point equations: ĥk = σ( ∑ ij Wijkvim̂j + ∑ i Phikvi) (11) m̂j = σ( ∑ ik Wijkviĥk + ∑ i Pmij vi + ∑ l Ujlêl) (12) êl = exp( ∑ j Ujlm̂j)∑ l′ exp( ∑ j Ujl′m̂j) (13) The model is trained to maximize the hybrid objective logP (v, e) + η logP (e|v) (Larochelle & Bengio, 2008).”
A Survey of Multi-View Representation Learning (2016, neutral): “The joint distribution over the multi-modal input can be written as: P (vm,vt; θ) = ∑ h (2) m ,h (2) t ,h (3) P ( h(2)m ,h (2) t ,h (3) )(∑ h (1) m P ( vm, h(1)m ,h (2) m ))(∑ h (1) t P ( vt,h (1) t ,h (2) t )) (43) Like RBM, exact maximum likelihood learning in this model is also intractable, while”
Learning Algorithms for the Classification Restricted Boltzmann Machine (2012, methodology): “We thank Dumitru Erhan for discussions about sparse RBMs. Earlier versions of some of the material presented here appeared in the ICML 2008 conference proceedings (Larochelle and Bengio, 2008) and on arXiv (Mandel et al., 2011b).”
A topological insight into restricted Boltzmann machines (2016, methodology): “Examples are density estimation to model human choice (Osogami and Otsuka, 2014), collaborative filtering (Salakhutdinov et al, 2007), information retrieval (Gehler et al, 2006), or multi-class classification (Larochelle and Bengio, 2008).”
Network anomaly detection with the restricted Boltzmann machine (2013, methodology): “The discriminative RBM is a promising tool in statistical machine learning [22].”