Dropout as a Bayesian Approximation : Insights and Applications

By Yg, UK CAM.AC.. 2015

Subfields: Optimization & Training, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

In Probabilistic & Bayesian Methods

Citations in the corpus

A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference (2019, background): “Gal and Ghahramani [14] in 2015 provided a theoretical framework for modelling Bayesian uncertainty.”
Dropout distillation (2016, methodology): “…view of dropout performing stochastic gradient descent of some regularized loss function (Baldi & Sadowski, 2013), or as an adaptive L2 regularizer (Wager et al., 2013), or more recently as an approximation to a well-known Bayesian model, namely the deep Gaussian process (Gal & Ghahramani, 2015a).”
Robustly representing inferential uncertainty in deep neural networks through sampling (2016, methodology): “(Gal & Ghahramani, 2015) detailed how dropping units was equivalent to sampling weights from a Bernoulli-based variational distribution and that in order to make a DNN with dropout Bayesian, sampling should be used during both training and inference.”
A Survey on Bayesian Deep Learning (2016, background): “Regularization techniques such as weight decay and dropout [103] are shown to be effective in improving performance of neural networks and they both have Bayesian interpretations [22].”
Ieee Transactions on Knowledge and Data Engineering 1 towards Bayesian Deep Learning: a Framework and Some Existing Methods (unknown): “Regularization techniques such as weight decay and dropout [57] are shown to be effective in improving performance of neural networks and they both have Bayesian interpretations [15].”
Uncertainty in Deep Learning (2016, unknown): “Some of the work in this thesis was previously presented in [Gal, 2015; Gal and Ghahramani, 2015a,b,c,d, 2016a,b,c; Gal et al., 2016], but this thesis contains many new pieces of work as well.”
Towards Bayesian Deep Learning: A Survey (2016, background): “Regularization techniques like weight decay and dropout [51] are shown to be effective in improving performance of neural networks and they both have Bayesian interpretations [13].”
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation (2015, background): “We are also interested in estimating the model uncertainty for predictions from deep segmentation architectures [68], [69].”
Representation of uncertainty in deep neural networks through sampling (2016, methodology): “Gal & Ghahramani (2015) detailed how dropping units was equivalent to sampling weights from a Bernoulli-based variational distribution and that in order to make a DNN with dropout Bayesian, sampling should be used during both training and inference.”