Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation

By Alireza Mehrtash, William M. Wells, Clare M. Tempany, Purang Abolmaesumi, Tina Kapur. IEEE Transactions on Medical Imaging, 2019

390 citations recorded by the source corpus.

Fully convolutional neural networks (FCNs), and in particular U-Nets, have achieved state-of-the-art results in semantic segmentation for numerous medical imaging applications. Moreover, batch normalization and Dice loss have been used successfully to stabilize and accelerate training. However, these networks are poorly calibrated i.e. they tend to produce overconfident predictions for both correct and erroneous classifications, making them unreliable and hard to interpret. In this paper, we study predictive uncertainty estimation in FCNs for medical image segmentation. We make the following contributions: 1) We systematically compare cross-entropy loss with Dice loss in terms of segmentation quality and uncertainty estimation of FCNs; 2) We propose model ensembling for confidence calibration of the FCNs trained with batch normalization and Dice loss; 3) We assess the ability of calibrated FCNs to predict segmentation quality of structures and detect out-of-distribution test examples. We conduct extensive experiments across three medical image segmentation applications of the brain, the heart, and the prostate to evaluate our contributions. The results of this study offer considerable insight into the predictive uncertainty estimation and out-of-distribution detection in medical image segmentation and provide practical recipes for confidence calibration. Moreover, we consistently demonstrate that model ensembling improves confidence calibration.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ed8d8ef1d9b917428319d06b1aa5696d444d8f66

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Detection and Retrieval of Out-of-Distribution Objects in Semantic Segmentation (2020, methodology): “In [22] the authors propose to use ensembles of models to calibrate the prediction confidence.”
TransMorph: Transformer for unsupervised medical image registration (2021, neutral): “The applications of Bayesian deep learning in medical imaging expands on image denoising (Tölle et al., 2021; Laves et al., 2020b) and image segmentation (DeVries and Taylor, 2018; Baumgartner et al., 2019; Mehrtash et al., 2020).”
Local Temperature Scaling for Probability Calibration (2020, methodology): “, via model ensembling [47] or multi-task learning [31]); (3) Most methods are designed to work for binary classifications and approach multi-class problems by a decomposition into k one-vs-rest binary calibrations (where k denotes the number of classes).”
Recent advances and clinical applications of deep learning in medical image analysis (2021, background): “…training (Xue et al., 2018; Zhang et al., 2020b), attention mechanism (Jetley et al., 2018; Anderson et al., 2018; Oktay et al., 2018; Nie et al., 2018; Sinha and Dolz, 2021), and uncertainty estimation (Wang et al., 2019; Yu et al., 2019; Baumgartner et al., 2019; Mehrtash et al., 2020).”
A survey of uncertainty in deep neural networks (2021, methodology): “Following several works based on deep ensembles [31], [275] and BNNs, [272], [209], [204] also compared their performance to other methods based on the resulting calibration.”