What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
By Alex Kendall, Yarin Gal. 2017
6,295 citations recorded by the source corpus.
There are two major types of uncertainty one can model. Aleatoric uncertainty captures noise inherent in the observations. On the other hand, epistemic uncertainty accounts for uncertainty in the model - uncertainty which can be explained away given enough data. Traditionally it has been difficult to model epistemic uncertainty in computer vision, but with new Bayesian deep learning tools this is now possible. We study the benefits of modeling epistemic vs. aleatoric uncertainty in Bayesian deep learning models for vision tasks. For this we present a Bayesian deep learning framework combining input-dependent aleatoric uncertainty together with epistemic uncertainty. We study models under the framework with per-pixel semantic segmentation and depth regression tasks. Further, our explicit uncertainty formulation leads to new loss functions for these tasks, which can be interpreted as learned attenuation. This makes the loss more robust to noisy data, also giving new state-of-the-art results on segmentation and depth regression benchmarks.
Subfields: Artificial Intelligence, Safety, Robustness & Interpretability
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Yoshua Bengio
- Quoc V. Le
- Mohit Bansal
- Percy Liang
- Dacheng Tao
- Trevor Darrell
- Jianfeng Gao
In Safety, Robustness & Interpretability
- Wojciech Samek
- Adrian Weller
- Nicolas Papernot
- Nicholas Carlini
- Been Kim
- K. Müller
- Grgoire Montavon
- Lars Kai Hansen
Citations in the corpus
- Scalable agent alignment via reward modeling: a research direction (2018, background): “Bayesian methods untangle irreducible uncertainty from ‘epistemic’ uncertainty about which parameters are correct, which decreases with the amount of data (Kendall & Gal, 2017); this distinction can help with active learning (Gal et al., 2017b).”
- Evidential Deep Learning to Quantify Classification Uncertainty (2018, background): “Our next proposition provides a guarantee for this preferable behavior pattern, which is known in the uncertainty modeling literature as learned loss attenuation [16].”
- A General Framework for Uncertainty Estimation in Deep Learning (2019, methodology): “Similarly to all sampling-based methods [3], [17], the main limitation of our approach is that, in order to generate σtot, we need several network forward passes for each input.”
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020, methodology): “stributional shift and facilitating generalization may find use in offline RL algorithms, including techniques from causal inference (Schölkopf, 2019), uncertainty estimation (Gal and Ghahramani, 2016; Kendall and Gal, 2017), density estimation and generative modeling (Kingma et al., 2014), distributi”
- Decentralized Bayesian Learning over Graphs (2019, neutral): “The posterior probability P ( y ) in Bayesian Deep Learning literature [4, 26, 27], is interpreted as the confidence of agent i in predicting y as the true label.”
- A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference (2019, background): “[9] [30].”
- A Simple Baseline for Bayesian Uncertainty in Deep Learning (2019, background): “From a practical perspective, these approaches are quite appealing as they only require ensembling dropout predictions at test time, and they were succesfully applied to several downstream tasks [21, 37].”
- Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “There exist several sources of uncertainty in ML applications [37].”
- Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI (2020, background): “, through stochastic behavior) while epistemic uncertainty (ambiguity) is the uncertainty due to limited data or knowledge.(15,33,34) For example, an uncertainty-aware image classifier should exhibit high aleatoric uncertainty for images that are similar to those it was trained on, but that do not c”
- On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, background): “In fact, the restrained access to only a small portion of past tasks increases the epistemic uncertainty [38] of the model: as a consequence, we expect the decision surfaces tied to past classes to slowly erode everywhere, with the exception of certain input regions i.”
- Do not Forget to Attend to Uncertainty while Mitigating Catastrophic Forgetting (2021, methodology): “However, [8, 14] proposed a dropout based method to approximate the posterior.”
- Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors (2020, background): “In principle, BNNs can permit graceful failure, signalling when a model does not know what to predict (Kendall & Gal, 2017; Dusenberry et al., 2019), and can also generalize better to out-of-distribution examples (Louizos & Welling, 2017; Malinin & Gales, 2018).”
- Calibrating Deep Convolutional Gaussian Processes (2018, background): “…( CNN s) in increasingly popular pieces of technology such as self driving cars and medical imaging, where decision-making under uncertainty is fundamental, has brought attention to the ability of these learning architectures to accurately quantify the uncertainty in their predictions [14, 8].”
- Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection (2018, methodology): “This dropout sampling method has been applied to active learning for cancer diagnosis [25], semantic segmentation [26], [27] and image object detection for openset problem [28].”
- Predictive Uncertainty Estimation via Prior Networks (2018, methodology): “This technique has been successfully applied to tasks in computer vision [19, 20].”
- Lightweight Probabilistic Deep Networks (2018, background): “[28] combine model uncertainty from Monte Carlo sampling, termed epistemic uncertainty, with noise from the observation model, termed aleatoric, het-eroscedastic uncertainty.”