Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding

By Alex Kendall, Vijay Badrinarayanan, Roberto Cipolla. 2015

1,141 citations recorded by the source corpus.

We present a deep learning framework for probabilistic pixel-wise semantic segmentation, which we term Bayesian SegNet. Semantic segmentation is an important tool for visual scene understanding and a meaningful measure of uncertainty is essential for decision making. Our contribution is a practical system which is able to predict pixel-wise class labels with a measure of model uncertainty. We achieve this by Monte Carlo sampling with dropout at test time to generate a posterior distribution of pixel class labels. In addition, we show that modelling uncertainty improves segmentation performance by 2-3% across a number of state of the art architectures such as SegNet, FCN and Dilation Network, with no additional parametrisation. We also observe a significant improvement in performance for smaller datasets where modelling uncertainty is more effective. We benchmark Bayesian SegNet on the indoor SUN Scene Understanding and outdoor CamVid driving scenes datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation (2015, methodology): “In future, we would also like to estimate the uncertainty of the labelling using Bayesian neural network techniques [64].”
An overview of deep learning in medical imaging focusing on MRI (2018, background): “In addition to producing valuable measures that function as uncertainty measures [65,355,356], these techniques can also lessen deep neural networks susceptibility to adversarial attacks [354,357].”
Deep Reinforcement Learning framework for Autonomous Driving (2017, background): “Deep learning models are able to learn complex feature representations from raw input data, omitting the need for hand-crafted features [15] [2] [7].”
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? (2017, methodology): “Following [22], we use 50 Monte Carlo dropout samples.”
Dropout Inference in Bayesian Neural Networks with Alpha-divergences (2017, methodology): “These have been used for machine vision and medical applications (Kendall & Cipolla, 2016; Kendall et al., 2015; Angermueller & Stegle, 2015; Yang et al., 2016).”
Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection (2018, methodology): “This dropout sampling method has been applied to active learning for cancer diagnosis [25], semantic segmentation [26], [27] and image object detection for openset problem [28].”
Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks (2017, methodology): “This means that Bayesian techniques for NNs are often only used near the top of the network [56, 30] with Kendall et al. [30, §4.1] arguing that the lower layers of a DNN for images may not beneﬁt as much from a Bayesian treatment as they are mainly simple edge and corner detectors.”
Concrete Dropout (2017, methodology): “Interestingly, we observe that Concrete dropout learns a different pattern to manual dropout tuning results [21].”
A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, methodology): “corroborate the decision to gain awareness of the unknown. In deep networks this could be achieved by assessing the variations of stochastic forward passes through a neural network with dropout [38], [146], [147], as a variational Bayesian approximation to a distribution on the weights [37], or by e”
Concrete Problems for Autonomous Vehicle Safety: Advantages of Bayesian Deep Learning (2017, positive): “Most algorithms make hard-classifications, however uncertainty can be captured with semantic segmentation algorithms which use Bayesian deep learning [Kendall et al., 2015].”
The Importance of Skip Connections in Biomedical Image Segmentation (2016, background): “However, we found that the border continuity can be improved for models with the Dice loss by implicit model averaging over output samples drawn at test time, using dropout [10] (Figure 2(d)).”
Learning 3D Object Categories by Looking Around Them (2017, methodology): “Following [18], for training it uses the BerHu depth loss and a dropout layer, which allows it to produce a confidence score of the depth measurements at test time using the sampling technique of [17, 9].”
A Probabilistic U-Net for Segmentation of Ambiguous Images (2018, background): “Our ‘Dropout U-Net’ baselines follow the Bayesian segnet’s [7] proposition: we dropout the activations of the respective incoming layers of the three inner-most encoder and decoder blocks with a dropout probability of p = 0.”
A Scalable Hierarchical Gaussian Process Classifier (2019, background): “As another example, in an autonomous vehicle system, the uncertainty of object segmentation result can be used to calculate the likelihood of critical events, and results in different behavioral decisions [10].”
Detection and Retrieval of Out-of-Distribution Objects in Semantic Segmentation (2020, methodology): “Works in the field of uncertainty or confidence estimation include Bayesian methods [8, 15, 38], ensemble methods [17] as well as approaches that acquire information from intermediate layers of the network or from its predictions to train a second model that serves as confidence estimator [6, 11, 23”