Transforming Neural-Net Output Levels to Probability Distributions

By John S. Denker, Yann LeCun. 1990

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, methodology): “Laplace approximation and GP inference with DNN Laplace approximation has a long history in GP and NN literature (Tierney et al., 1989; Denker and LeCun, 1991; Rasmussen and Williams, 2006; MacKay, 1992; Ritter et al., 2018), and the theoretical connection between a Laplace-approximated DNN and GP…”
A General Framework for Uncertainty Estimation in Deep Learning (2019, background): “proaches [3], [25]–[28], we represent this uncertainty by placing a distribution over the neural network weights, ω.”
A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference (2019, background): “Denker and LeCun [7], and MacKay [44] investigated the posterior probability distributions of neural networks and treated the search in the model space (the space of architectures, weight decay, regularizers, etc..) as an inference problem and tried to solve it using Laplace approximations.”
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? (2017, neutral): “Such a model is referred to as a Bayesian neural network (BNN) [9–11].”
Dropout Inference in Bayesian Neural Networks with Alpha-divergences (2017, background): “Originating in the 90s (Neal, 1995; MacKay, 1992; Denker & LeCun, 1991), Bayesian neural networks (BNNs) in particular have started gaining in popularity again (Graves, 2011; Blundell et al., 2015; Hernandez-Lobato & Adams, 2015).”
Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding (2015, background): “In general, this posterior distribution is not tractable, therefore we need to approximate the distribution of these weights [9].”
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam (2018, neutral): “M L ] 2 A ug 2 01 8 rithms such as MCMC methods (Neal, 1995), Laplace’s method (Denker & Lecun, 1991), and variational inference (Hinton & Van Camp, 1993; Barber & Bishop, 1998).”
A survey of uncertainty in deep neural networks (2021, methodology): “[122] have pioneered the Laplace approximation for neural networks in 1990s, and several modern methods provide an extension to deep neural networks [192], [193], [63], [84].”
Laplace Redux - Effortless Bayesian Deep Learning (2021, background): “The most lightweight is a diagonal factorization which ignores off-diagonal elements [42, 43].”
Uncertainty in Deep Learning (2016, methodology): “MacKay [1992b] advocated the use of model evidence for model comparison, and obtained this quantity following the approximation of [Denker and LeCun, 1991].”
Aleatoric and Epistemic Uncertainty in Machine Learning: A Tutorial Introduction (2019, methodology): “To capture this kind of epistemic uncertainty, Bayesian neural networks (BNNs) have been proposed as a Bayesian extension of deep neural networks (Denker and LeCun, 1991; Kay, 1992; Neal, 2012).”