Evaluating Bayesian Deep Learning Methods for Semantic Segmentation

By Jishnu Mukhoti, Yarin Gal. 2018

268 citations recorded by the source corpus.

Deep learning has been revolutionary for computer vision and semantic segmentation in particular, with Bayesian Deep Learning (BDL) used to obtain uncertainty maps from deep models when predicting semantic classes. This information is critical when using semantic segmentation for autonomous driving for example. Standard semantic segmentation systems have well-established evaluation metrics. However, with BDL's rising popularity in computer vision we require new metrics to evaluate whether a BDL method produces better uncertainty estimates than another method. In this work we propose three such metrics to evaluate BDL models designed specifically for the task of semantic segmentation. We modify DeepLab-v3+, one of the state-of-the-art deep neural networks, and create its Bayesian counterpart using MC dropout and Concrete dropout as inference techniques. We then compare and test these two inference techniques on the well-known Cityscapes dataset using our suggested metrics. Our results provide new benchmarks for researchers to compare and evaluate their improved uncertainty quantification in pursuit of safer semantic segmentation.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

A Simple Baseline for Bayesian Uncertainty in Deep Learning (2019, methodology): “Markov chain Monte Carlo (MCMC) was at one time a gold standard for inference with neural networks, through the Hamiltonian Monte Carlo (HMC) work of Neal (1996). However, HMC requires full gradients, which is computationally intractable for modern neural networks.”
Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels (2022, background): “Model Uncertainty in computer vision is mostly measured by Bayesian deep learning approaches [14, 25, 31].”
Uncertainty-Aware Audiovisual Activity Recognition Using Deep Bayesian Variational Inference (2018, background): “Figure 4 shows accuracy vs uncertainty confusion matrix (proposed in [31] for semantic segmentation), which A reliable model will provide higher A v U score.”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, methodology): “[10], optical flow estimation and motion prediction [249], [410], [411], human pose estimation and pedestrian localization [412], [413], [305], person re-identification and face recognition [414], [415], [416], camera re-localization [397], avoiding adversarial attacks [417], [418], during the years”
TRADI: Tracking deep neural network weight distributions (2019, background): “placing Dropout only next to a specific set of layers [21, 32].”
A survey of uncertainty in deep neural networks (2021, unknown): “potential applications [137], [189], [216], [208].”
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision (2019, methodology): “Among such scalable methods, MC-dropout [12, 25, 24, 36] and ensembling [27, 9, 23] are clearly the most widely employed, due to their demonstrated effectiveness and simplicity.”