Variational Learning in Nonlinear Gaussian Belief Networks

By Brendan J. Frey, Geoffrey E. Hinton. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A General Framework for Uncertainty Estimation in Deep Learning (2019, methodology): “We refer the reader to [6], [5], [24] for the details of the propagation formulas.”
Lightweight Probabilistic Deep Networks (2018, methodology): “However, we build upon the layerwise transformations of Gaussian distributions by Hin-ton and Frey [12] in the context of nonlinear Gaussian belief networks, and repurpose them here for the representation of activation uncertainties in deep supervised learning.”
Nonlinear Statistical Learning with Truncated Gaussian Graphical Models (2016, methodology): “To overcome the linearity of GGMs, Frey (1997), Hinton & Ghahramani (1997), and Frey & Hinton (1999) proposed to apply nonlinear transformations to Gaussian hidden variables.”
Stochastic Back-propagation and Variational Inference in Deep Latent Gaussian Models (2014, methodology): “00 NLGBN (Frey & Hinton, 1999) 95 .”
Learning the Structure of Deep Sparse Graphical Models (2009, methodology): “To achieve a wide range of possible behaviors, we use the nonlinear Gaussian belief network (NLGBN) (Frey, 1997; Frey and Hinton, 1999) framework.”
Principal Manifolds and Probabilistic Subspaces for Visual Recognition (2002, methodology): “Index TermsÐSubspace techniques, PCA, ICA, Kernel PCA, Probabilistic PCA, learning, density estimation, face recognition. æ”
Rectified Factor Networks (2015, unknown): “For example, generative models with rectified priors, like rectified factor analysis, have zero posterior probability for negative values, therefore their means are positive and not sparse [10, 11].”
Learning Deep Sigmoid Belief Networks with Data Augmentation (2015, background): “The probit function and the rectified linearity are also considered in the nonlinear Gaussian belief network (Frey and Hinton, 1999).”
Variational learning for rectified factor analysis (2007, methodology): “2, a fixed-form variational solution exists for the linear model with rectification nonlinearity, as developed in [9] for nonlinear belief networks.”
Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation (2019, background): “The output of a ReLU activation that receives a Gaussian input Z ∼ N ( µ , σ 2 ) is a rectiﬁed Gaussian distribution (Socci et al., 1998) whose mean and variance can be derived analytically (Frey & Hinton, 1999).”
Stochastic Backpropagation and Approximate Inference in Deep Generative Models (2014, methodology): “Model − ln p(v) Factor Analysis 106.00 NLGBN (Frey & Hinton, 1999) 95.80 Wake-Sleep (Dayan, 2000) 91.3 DLGM diagonal covariance 87.30 DLGM rank-one covariance 86.60 Results below from Uria et al. (2014) in a grid between -5 and 5.”