Varieties of Helmholtz Machine

By Peter S. Dayan, Geoffrey E. Hinton. 1996

Subfields: Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Probabilistic & Bayesian Methods

Citations in the corpus

Neural Variational Inference for Text Processing (2015, methodology): “In Figure 4, we compare the effectiveness of the latent attention mechanism (NASM) and its deterministic counter-part (LSTM+Att) by visualising the attention scores on the answer sentences.”
Deep learning in neural networks: An overview (2014, background): “The Helmholtz Machine (Dayan & Hinton, 1996; Dayan, Hinton, Neal, & Zemel, 1995) may be viewed as an unsupervised (Section 5.6.4) variant thereof (Peter Dayan, personal communication, 1994).”
Reweighted Wake-Sleep (2014, background): “It was known that this approach results in a gradient estimate of very high variance for the recognition network q (Dayan and Hinton, 1996).”
What do we mean by prediction in language comprehension? (2015, background): “Actively generative models have also been instantiated in some neural networks (e.g. Dayan & Hinton, 1996; Dayan, Hinton, Neal, & Zemel, 1995; Hinton, 2007, see also forward models in the motor literature, e.g. Jordan & Rumelhart, 1992).”
An Interoceptive Predictive Coding Model of Conscious Presence (2011, methodology): “…this minimization are based on so-called “variational Bayes” worked out by (Neal and Hinton, 1998) among others; these algorithms have plausible neurobiological implementations, at least in cortical hierarchies (Hinton and Dayan, 1996; Lee and Mumford, 2003; Friston et al., 2006; Friston, 2009).”
Smart Regularization of Deep Architectures (2015, background): “However, this estimator is notorious for its high variance [6, 23, 24].”
Merging chrominance and luminance in early, medium, and late fusion using Convolutional Neural Networks (2015, methodology): “More information about the HM and the wake-sleep algorithm can be found in [Dayan and Hinton, 1996; Dayan, 2000; Kirby, 2006].”
Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning (2022, unknown): “1We note that the DCCP action decomposition is related to certain hierarchical approaches to RL, such as feudal RL [4, 7], and to mixtures-of-experts [12, 16], though our DM RL methods do not influence the training of the providers.”
Bidirectional Helmholtz Machines (2015, methodology): “, 1995; Dayan et al., 1995), a concept was introduced that proposed to not only fit a powerful but intractable generative model p(x,h) to the training data, but also to jointly train a parametric approximate inference model q(h|x). The distribution q is used to perform approximate inference over the”