Modelling uncertainty in deep learning for camera relocalization

By Alex Kendall, Roberto Cipolla. 2015

598 citations recorded by the source corpus.

We present a robust and real-time monocular six degree of freedom visual relocalization system. We use a Bayesian convolutional neural network to regress the 6-DOF camera pose from a single RGB image. It is trained in an end-to-end manner with no need of additional engineering or graph optimisation. The algorithm can operate indoors and outdoors in real time, taking under 6ms to compute. It obtains approximately 2m and 6° accuracy for very large scale outdoor scenes and 0.5m and 10° accuracy indoors. Using a Bayesian convolutional neural network implementation we obtain an estimate of the model's relocalization uncertainty and improve state of the art localization accuracy on a large scale outdoor dataset. We leverage the uncertainty measure to estimate metric relocalization error and to detect the presence or absence of the scene in the input image. We show that the model's uncertainty is caused by images being dissimilar to the training dataset in either pose or appearance.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Dropout Inference in Bayesian Neural Networks with Alpha-divergences (2017, methodology): “These have been used for machine vision and medical applications (Kendall & Cipolla, 2016; Kendall et al., 2015; Angermueller & Stegle, 2015; Yang et al., 2016).”
Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding (2015, methodology): “This technique has seen success in modelling uncertainty for camera relocalisation [19].”
Concrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning (2017, methodology): “For this task as well there exist computer vision techniques which use Bayesian deep learning [Kendall and Cipolla, 2016].”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, background): “uncertainty quantity technique [397], [43], [249], [420], [394], [413], [10], [411], [418], applicable on various deep learning architectures.”
Aggressive Deep Driving: Combining Convolutional Neural Networks and Model Predictive Control (2017, methodology): “An alternative method to providing absolute position uses deep neural networks to directly regress a position estimate in an area previously visited [13].”
Image-Based Localization Using LSTMs for Structured Feature Correlation (2016, methodology): “Scene Area or Volume Active Search (w/o) [43] Active Search (w/) [43] PoseNet [23] Bayesian PoseNet [22] Proposed + Improvement(pos,ori)”
Relaxed Softmax: Efficient Confidence Auto-Calibration for Safe Pedestrian Detection (2018, methodology): “Kendall and Cipolla [14] estimate uncertainty in a camera pose regression problem, exploiting Monte Carlo sampling.”
Uncertainty-Aware Audiovisual Activity Recognition Using Deep Bayesian Variational Inference (2018, methodology): “Bayesian deep learning framework with Monte Carlo (MC) dropout approximate inference [13] is used in visual scene understanding applications including camera relocalization [25], semantic segmentation [24] and depth regression [26].”
Learning to Navigate the Energy Landscape (2016, background): “We also demonstrate results on RGB camera relocalization, and refer the reader to [22] for a tour of the literature.”
Uncertainty in Deep Learning (2016, background): “4) have been developed in recent literature [Bulò et al., 2016; Kendall and Cipolla, 2016], with specific attention paid to model uncertainty in image segmentation [Furnari et al.”
Geometric Loss Functions for Camera Pose Regression with Deep Learning (2017, methodology): “Later work has extended PoseNet to use RGB-D input [25], learn relative ego-motion [31], improve the context of features [54], localise over video sequences [8] and interpret relocalisation uncertainty with Bayesian Neural Networks [19].”
Aggressive Deep Driving: Model Predictive Control with a CNN Cost Model (2017, methodology): “An alternative method to providing absolute position uses deep neural networks to directly regress a position estimate in an area previously visited [13].”
A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Tasks (2019, background): “BDL has already been demonstrated to play a crucial role in applications such as medical diagnostics [30, 16, 5, 49] (see Figure 1), computer vision [21, 18, 17], in the sciences [31, 34], and autonomous driving [1, 15, 21, 18, 19].”
Dropout Sampling for Robust Object Detection in Open-Set Conditions (2017, methodology): “y deep neural network to become Bayesian by simply enabling the dropout layers during testing, as opposed to standard practice where dropout layers are only used during training. Recently, in [8] and [9], dropout sampling was used for uncertainty estimates on regression and image classiﬁcation tasks”
Geometric Consistency for Self-Supervised End-to-End Visual Odometry (2018, methodology): “[11] use dropout [20] as a means of bayesian approximation to recover covariance from relocalization estimates from a trained CNN.”
Understanding the Limitations of CNN-Based Absolute Camera Pose Regression (2019, background): “[12, 28] also reason about the uncertainty of the estimated poses.”
A Review and Comparative Study on Probabilistic Object Detection in Autonomous Driving (2020, background): “…in diﬀerent components of autonomous driving such as semantic segmentation [36]–[38], optical ﬂow [39], [40], depth estimation [35], [41]–[43], visual odometry [44]– [46], image annotation [47]–[50], visual tracking [51], trajectory prediction [52]–[55] and end-to-end perception [56]–[58].”