Connectionist Learning of Belief Networks

By Radford M. Neal. 1992

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Neural Variational Inference and Learning in Belief Networks (2014, positive): “Although the generality of Markov Chain Monte Carlo (MCMC) methods makes them straightforward to apply to models of this type (Neal, 1992), they tend to suffer from slow mixing and are usually too computationally expensive to be practical in all but the simplest models.”
Neural Autoregressive Distribution Estimation (2016, methodology): “Work on directed generative networks dates back to the original work on sigmoid belief networks (Neal, 1992) and the Helmholtz machine (Hinton et al., 1995; Dayan et al., 1995).”
Pixel Recurrent Neural Networks (2016, background): “The factorization turns the joint modeling problem into a sequence problem, where one learns to predict the next pixel given all the previously generated pixels.”
Generative Moment Matching Networks (2015, neutral): “Next there is the class of fully visible directed models such as fully visible sigmoid belief networks (Neal, 1992) and the neural autoregressive distribution estimator (Larochelle & Murray, 2011).”
Representation Learning: A Review and New Perspectives (2012, methodology): “Examples of this decomposition include: Principal Components Analysis (PCA) (Roweis, 1997; Tipping and Bishop, 1999), sparse coding (Olshausen and Field, 1996), sigmoid belief networks (Neal, 1992) and the newly introduced spike-and-slab sparse coding model (Goodfellow et al., 2011).”
Importance Weighted Autoencoders (2015, background): “Other models are deﬁned in terms of belief networks (Neal, 1992; Gregor et al., 2014).”
Generative Image Modeling Using Spatial LSTMs (2015, background): “One successful approach to building flexible yet tractable generative models has been to use fullyvisible belief networks [21, 27].”
MuProp: Unbiased Backpropagation for Stochastic Neural Networks (2015, methodology): “The gradient g = ∂L/∂θ is usually intractable to compute exactly, and must therefore be estimated using an estimator ĝ(x), i.e. g ≈ ∑m i=1 ĝ(xi)/m, where xi ∼ pθ(x) and m is the number of Monte Carlo samples.”
Lightweight Probabilistic Deep Networks (2018, background): “Perhaps among the ﬁrst probabilistic neural networks are the works relating undirected graphical models to neural networks via Boltz-mann machines [3, 24]; similarly, links are established for deep belief nets (DBN) via sigmoid belief nets [23, 41].”
Nonlinear Statistical Learning with Truncated Gaussian Graphical Models (2016, background): “The restricted Boltzmann machine (RBM) (Hinton, 2002) and sigmoid belief network (SBN) (Neal, 1992; Gan et al., 2015), as well as the deep networks built upon them (Salakhutdinov & Hinton, 2009; Hinton et al., 2006), are good examples of using latent variables to enhance modeling versatility while…”
Discrete Variational Autoencoders (2016, methodology): “…further restrict the neural network deﬁning the distribution over the observed variables x given the smoothing variables ζ to consist of a linear transformation followed by a pointwise logistic nonlinearity, analogous to a sigmoid belief network (SBN; Spiegelhalter & Lauritzen, 1990; Neal, 1992).”
Mean Field Theory for Sigmoid Belief Networks (1996, background): “The ease of interpretation a(cid:11)orded by this semantics explains the growing appeal of belief networks, now widely used as models of planning, reasoning, and uncertainty.”
Techniques for Learning Binary Stochastic Feedforward Neural Networks (2014, methodology): “Stochastic feedforward neural networks (SFNN) (Neal, 1990, 1992) have the advantage when the conditionals P ( y | x ) are more complicated.”
Herding Dynamic Weights for Partially Observed Random Field Models (2009, background): “Our “herding dynamics” does not require expensive operations such as exponentiation and is fully deterministic.”
High-Dimensional Probability Estimation with Deep Density Models (2013, background): “Directed belief networks [Pearl, 1988, Neal, 1992, Adams et al., 2010], on the other hand, enable one to specify a priori marginals of hidden variables and are easily normalized, but require costly inference procedures.”
Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration (2017, methodology): “This idea of modeling the joint probability distribution of outputs by casting it as a product of conditional distributions is used in Pixel RNNs [43], Neural Autoregressive Distribution Estimator (NADE) [35], and fully visible neural networks [44], [45].”
The Generalized Reparameterization Gradient (2016, background): “…models (Carbonetto et al., 2009; Paisley et al., 2012; Ranganath et al., 2014; Titsias and Lázaro-Gredilla, 2014), deep neural networks (Neal, 1992; Hin-ton et al., 1995; Mnih and Gregor, 2014; Kingma and Welling, 2014), and probabilistic programming (Wingate and Weber, 2013;…”