The Rectified Gaussian Distribution

By Nicholas D. Socci, Daniel D. Lee, H. Sebastian Seung. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Lawrence Carin
2. Chunyuan Li
3. Yu Zhang
4. Changyou Chen
5. Yoshua Bengio
6. Ivan Titov
7. Qiang Yang
8. Zhe Gan

Citations in the corpus

• Nonlinear Statistical Learning with Truncated Gaussian Graphical Models (2016, background): “It should be emphasized that the approach proposed here is different from those in (Socci et al., 1998; Downs et al., 1999), which constrain the target (observed) variables to be nonnegative, without using latent variables; the nonnegative constraint in those approaches relaxes the convex function i”
• Interpretable Neural Predictions with Differentiable Binary Variables (2019, methodology): “We propose to replace Bernoulli variables by rectified continuous random variables (Socci et al., 1998), for they exhibit both discrete and continuous behaviour.”
• The Nonnegative Boltzmann Machine (1999, background): “In this case, the maximum entropy distribution for nonnegative data with known first and second order statistics is described by a distribution previously called the “rectified Gaussian” distribution [3]: if if any (2)”
• Variational learning for rectified factor analysis (2007, result): “Indeed, differently from the initial use of this term [11] (where it meant a multivariate distribution over the positive domain), in [12], RN has been termed as rectified Gaussian, whereas other, more recent works [6,7], including this paper, refer to NR by the same term.”
• Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation (2019, background): “The output of a ReLU activation that receives a Gaussian input Z ∼ N (μ,σ(2)) is a rectified Gaussian distribution (Socci et al., 1998) whose mean and variance can be derived analytically (Frey & Hinton, 1999).”