A General Framework for Uncertainty Estimation in Deep Learning

By Antonio Loquercio, Mattia Segu, Davide Scaramuzza. IEEE Robotics and Automation Letters, 2019

354 citations recorded by the source corpus.

Neural networks predictions are unreliable when the input sample is out of the training distribution or corrupted by noise. Being able to detect such failures automatically is fundamental to integrate deep learning algorithms into robotics. Current approaches for uncertainty estimation of neural networks require changes to the network and optimization process, typically ignore prior knowledge about the data, and tend to make over-simplifying assumptions which underestimate uncertainty. To address these limitations, we propose a novel framework for uncertainty estimation. Based on Bayesian belief networks and Monte-Carlo sampling, our framework not only fully models the different sources of prediction uncertainty, but also incorporates prior data information, e.g. sensor noise. We show theoretically that this gives us the ability to capture uncertainty better than existing methods. In addition, our framework has several desirable properties: (i) it is agnostic to the network architecture and task; (ii) it does not require changes in the optimization process; (iii) it can be applied to already trained architectures. We thoroughly validate the proposed framework through extensive experiments on both computer vision and control tasks, where we outperform previous methods by up to 23% in accuracy. The video available at https://youtu.be/X7n-bRS5vSM shows qualitative results of our experiments. The project's code is available at: https://tinyurl.com/s3nygw7.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

View Paper: https://arxiv.org/abs/1907.06890

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

End-to-End Autonomous Driving: Challenges and Frontiers (2023, background): “Loquercio et al. [232] and Filos et al. [234] propose capturing epistemic uncertainty with an .”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, background): “uncertainty quantity technique [397], [43], [249], [420], [394], [413], [10], [411], [418], applicable on various deep learning architectures.”
A survey of uncertainty in deep neural networks (2021, methodology): “medical image analysis [1], [2], [3], [4], [5], [6] or autonomous vehicle control [7], [8], [9], [10], their deployment in mission-”
SalsaNext: Fast, Uncertainty-Aware Semantic Segmentation of LiDAR Point Clouds (2020, positive): “Following the work of [29], we further computed the”
A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving (2020, background): “2), optical flow [43], [60], depth estimation [50], [57], [61], [62], visual odometry [35], [63], [64], image annotation [41], [65]– [67], visual tracking [52], trajectory prediction [49], [68]–[70] and end-to-end perception [47], [71], [72].”
OpenGAN: Open-Set Recognition via Open Data Generation (2021, neutral): “Bayesian neural networks estimate uncertainties via Monte Carlo estimates (MCdrop) [17, 31].”
Machine learning and deep learning—A review for ecologists (2022, methodology): “New methods such as Bayesian neural networks have paved the way to obtain uncertainty and prediction intervals for DL models, bringing ML algorithms closer to statistical models (Ashukha et al., 2021 ; Loquercio et al., 2020 ).”