Multi-Prediction Deep Boltzmann Machines

By Ian J. Goodfellow, Mehdi Mirza, Aaron C. Courville, Yoshua Bengio. 2013

139 citations recorded by the source corpus.

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Generative Adversarial Nets (2014, background): “Essentially, one can use adversarial nets to implement a stochastic extension of the deterministic MP-DBM [10].”
Deep Generative Stochastic Networks Trainable by Backprop (2013, methodology): “This has been exploited in order to obtain much better samples using the associated GSN Markov chain than by sampling from the corresponding DBM (Goodfellow et al., 2013).”
Explaining and Harnessing Adversarial Examples (2014, background): “The MP-DBM (Goodfellow et al., 2013a) provides a good model to test this hypothesis.”
Conditional Generative Adversarial Nets (2014, background): “Such conditioning could be based on class labels, on some part of data for inpainting like [5], or even on data from different modality.”
Generative adversarial networks (2014, positive): “ subset of the indices of x by training a family of conditional models that share parameters. Essentially, one can use adversarial nets to implement a stochastic extension of the deterministic MP-DBM [11]. 4. Semi-supervised learning: features from the discriminator or inference net could improve pe”
Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks (2015, background): “…typically involve either training a generative model – parameterized, for example, by deep Boltzmann machines (e.g. Salakhutdinov & Hinton (2009), Goodfellow et al. (2013)) or by feed-forward neural networks (e.g. Bengio et al. (2014), Kingma et al. (2014)) –, or training autoencoder networks…”
Semi-supervised Learning with Ladder Networks (2015, methodology): “The Multi-prediction deep Boltzmann machine (MP-DBM) (Goodfellow et al. , 2013a) is a way to train a DBM with backpropagation through variational inference.”
A Deep and Tractable Density Estimator (2013, methodology): “Goodfellow et al. (2013) introduced a procedure to train deep Boltzmann machines by maximizing a variational approximation of their generalised pseudo likelihood.”
Deep Learning: Methods and Applications (2014, background): “Another prominent type of deep unsupervised models with generative capability is the deep Boltzmann machine or DBM [131, 315, 316, 348].”
OptNet: Differentiable Optimization as a Layer in Neural Networks (2017, background): “Among many other applications, these architectures are well-studied for generic classification and structured prediction tasks (Goodfellow et al., 2013; Stoyanov et al., 2011; Brakel et al., 2013; LeCun et al., 2006; Belanger & McCallum, 2016; Belanger et al., 2017; Amos et al., 2017); in vision for”
Qualitatively characterizing neural network optimization problems (2014, methodology): “lized with too large of random weights. This barrier is reasonably wide but not very tall. See Fig. 6 for details. To examine the behavior of SGD on generative models, we experimented with an MP-DBM (Goodfellow et al., 2013a). This model is useful for our purposes because it gets good performance as”
Training Deep Neural Networks on Noisy Labels with Bootstrapping (2014, background): “More recently, multi-prediction DBM training (Goodfellow et al., 2013) and Generative Stochastic Networks (Ben-gio & Thibodeau-Laufer, 2013) improved the performance and simpliﬁed the training of deep generative models, enabling training via backpropagation much like in standard deep supervised…”
Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification (2016, background): “retraining has become less signiﬁcant and even overshadowed (LeCun et al., 2015) in the ﬁeld. Several attempts (e.g., Ranzato & Szummer (2008); Larochelle & Bengio (2008); Sohn et al. (2013); Goodfellow et al. (2013)) had been made to couple the unsupervised and supervised learning in the sa”
Understanding Deep Architectures using a Recursive Convolutional Network (2013, methodology): “In contrast to the model we use, however, [15] trains unsupervised using contrastive divergence, while [6] is nonconvolutional and focuses on conditioning on random combinations of inputs and targets.”
Crafting adversarial input sequences for recurrent neural networks (2016, methodology): “used the method in [2] to craft adversarial samples on a multi-prediction deep Boltzmann machine [17], which uses recurrent connections to classify inputs of fixed size.”
Iterative Neural Autoregressive Distribution Estimator NADE-k (2014, methodology): “, 2000) as well as generalized pseudo-log-likelihood (Goodfellow et al., 2013), but have the disadvantage that sampling from the trained “stochastic fill-in” model requires a Markov chain (repeatedly resampling some subset of the values given the others).”
Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures (2014, methodology): “In [14] inference in a graphical model is implemented via an ensemble of unfolded inference algorithms trained to predict one held out variable given the others.”
New Era for Robust Speech Recognition (2017, methodology): “These include the delay-and-sum (DS) beamformer [13, 33, 37], the max-SNR beamformer [19, 37, 39], the minimum variance distortionless response (MVDR) beamformer [16, 22, 48], and the multichannel Wiener filter (MCWF) [14, 34].”