Learning Stochastic Feedforward Neural Networks

By Yichuan Tang, Ruslan Salakhutdinov. 2013

134 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Learning to generate chairs with convolutional neural networks (2014, methodology): “[25] developed an approach for training a generative model with variational inference by performing (stochastic) backpropagation through a latent Gaussian representation.”
Semi-supervised Learning with Deep Generative Models (2014, methodology): “The model used in this way also provides an alternative model to the stochastic feed-forward networks (SFNN) described by Tang and Salakhutdinov (2013).”
Attribute2Image: Conditional Image Generation from Visual Attributes (2015, background): “Tang and Salakhutdinov [38] proposed stochastic feed-forward neural networks (SFNNs) with binary stochastic neurons and trained them by maximizing the variational lower bound of the conditional likelihood via generalized EM algorithm.”
Neural Variational Inference for Text Processing (2015, neutral): “Hence empirically, ’how’ questions are harder to ’understand and answer’.”
MuProp: Unbiased Backpropagation for Stochastic Neural Networks (2015, methodology): “We do not consider the importance-sampling based approaches (Raiko et al., 2015; Tang & Salakhutdinov, 2013; Bornschein & Bengio, 2015; Burda et al., 2015) as these can be interpreted as optimizing a different objective (Burda et al., 2015) that requires sampling the latent variables multiple times.”
Variational Inference for Monte Carlo Objectives (2016, background): “We will now show that this approach does not bias the resulting estimator.”
Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks (2015, background): “The question of training neural networks with stochastic discrete-valued hidden units has a long history, starting with Boltzmann machines [36], with recent work studying how to deal with such units in a system trained using back-propagated gradients [37], [38], [39], [40].”
Techniques for Learning Binary Stochastic Feedforward Neural Networks (2014, methodology): “More recent work proposes simply drawing samples from P ( h | x ) during the feedforward phase (Hinton et al. , 2012; Bengio, 2013; Tang and Salakhutdinov, 2013).”
Reweighted Wake-Sleep (2014, methodology): “eep provides a less biased (asymptotically unbiased) estimator of the log-likelihood and of its gradient. Similar in spirit, but aimed at a structured output prediction task is the method proposed by Tang and Salakhutdinov (2013). The authors optimize the variational bound of the log-likelihood inst”
Stochastic Neural Networks for Hierarchical Reinforcement Learning (2016, methodology): “To learn the span of skills, we propose to use Stochastic Neural Networks (SNNs) (Neal, 1990; 1992; Tang & Salakhutdinov, 2013), a class of neural networks with stochastic units in the computation graph.”
Learning Temporal Transformations from Time-Lapse Videos (2016, background): “Image/Frame generation: Generative models have attracted extensive attention in machine learning [16,17,18,19,20,21].”
Under Review as a Conference Paper at Iclr 2016 Predicting Distributions with Linearizing Belief Networks (2015, background): “Different objectives can be employed to train their parameters: data likelihood, variational bounds (Tang & Salakhutdinov, 2013), or a distiguishibility criteria (Goodfellow et al., 2014).”
A survey of randomized algorithms for training neural networks (2016, background): “Multi-modal conditional distribution is proposed in [70] by using stochastic hidden variables rather than deterministic ones.”