Uncertainty in Deep Learning

By Yarin Gal. 2016

1,654 citations recorded by the source corpus.

Subfields: Safety, Robustness & Interpretability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

In Probabilistic & Bayesian Methods

Citations in the corpus

Weight Agnostic Neural Networks (2019, background): “The weight parameters of a BNN [3, 4, 26, 43, 68, 78] are not ﬁxed values, but sampled from a distribution.”
Decentralized Bayesian Learning over Graphs (2019, neutral): “The posterior probability P ( y ) in Bayesian Deep Learning literature [4, 26, 27], is interpreted as the conﬁdence of agent i in predicting y as the true label.”
Peer-to-peer Federated Learning on Graphs (2019, methodology): “In particular, we consider Variational Inference (VI) (Gal, 2016, Chapter 3) techniques which replace the Bayesian modelling marginalization with optimization.”
Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “Contrary to the popular belief, this approach is not a reliable estimator of confidence [39].”
Interpreting Deep Classifier by Visual Distillation of Dark Knowledge (2018, methodology): “To see this, we use the predictive entropy H [ π i ] as a conﬁdence measure, a common choice in information theory (Gal, 2016), to color points.”
An overview of deep learning in medical imaging focusing on MRI (2018, background): “By leaving dropout turned on also during inference one effectively performs variational inference [64, 58, 65].”
Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI (2020, background): “‘‘confident’’ to refer to epistemic, and ‘‘chance’’ or ‘‘probability’’ to refer to aleatoric uncertainty(35)—even if only subconsciously and despite their frequent conflation in mathematical modeling.(22,36) It is particularly important to understand epistemic uncertainty in the coalition scenario.”
Getting a CLUE: A Method for Explaining Uncertainty Estimates (2020, background): “Fortunately, there is a rich literature concerning approximations to these objects [1; 7; 8].”
Counterfactual Prediction with Deep Instrumental Variables Networks (2016, background): “11 Gal [2016] expands on the ideas of Gal and Ghahramani [2015].”
Bayesian Hypernetworks (2017, methodology): “Most recent work focuses on variational inference (Blundell et al., 2015; Gal, 2016), and restricts the variational posterior to a simple family of distributions, for instance a factorial Gaussian (Blundell et al., 2015).”
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? (2017, background): “For both settings uncertainty can be captured with Bayesian deep learning approaches – which offer a practical framework for understanding uncertainty with deep learning models [6].”
Dropout Inference in Bayesian Neural Networks with Alpha-divergences (2017, methodology): “The objective offers a limited intuitive interpretation which is difﬁcult to explain to non-experts, and of limited use for engineers (Gal, 2016, Section 2.2.2).”
Simple and Scalable Epistemic Uncertainty Estimation Using a Single Deep Deterministic Neural Network (2020, background): “This does not happen in practice, and its found that the only uncertainty that can reliably be captured by looking at the entropy of the softmax distribution is aleatoric uncertainty (Gal, 2016).”
Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks (2020, background): “A principal goal of uncertainty calibration is that learning machines and neural networks in particular, should assign low conﬁdence to test cases not explained well by the training data or prior information (Gal, 2016).”
Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection (2018, methodology): “Gal [24] propose a method that captures the uncertainty in BNNs at test time by sampling the network multiple times with dropout.”
Concrete Dropout (2017, methodology): “But to get well-calibrated uncertainty estimates it is necessary to adapt the dropout probability as a variational parameter to the data at hand [7].”
A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, methodology): “Using Bayesian methods to quantify and measure uncertainties of deep neural networks were proposed in [208].”
Towards Robust Evaluations of Continual Learning (2018, methodology): “To measure uncertainty, we use the mutual information between predictions on each task and the model posterior, following Gal (2016).”
Deep Neural Networks as Gaussian Processes (2017, background): “For conventional neural networks, capturing the uncertainty in a model’s predictions is challenging (Gal, 2016).”