Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks

By Guotai Wang, Wenqi Li, Michael Aertsen, Jan Deprest, Sébastien Ourselin, Tom Kamiel Magda Vercauteren. Neurocomputing, 2018

730 citations recorded by the source corpus.

Despite the state-of-the-art performance for medical image segmentation, deep convolutional neural networks (CNNs) have rarely provided uncertainty estimations regarding their segmentation outputs, e.g., model (epistemic) and image-based (aleatoric) uncertainties. In this work, we analyze these different types of uncertainties for CNN-based 2D and 3D medical image segmentation tasks at both pixel level and structure level. We additionally propose a test-time augmentation-based aleatoric uncertainty to analyze the effect of different transformations of the input image on the segmentation output. Test-time augmentation has been previously used to improve segmentation accuracy, yet not been formulated in a consistent mathematical framework. Hence, we also propose a theoretical formulation of test-time augmentation, where a distribution of the prediction is estimated by Monte Carlo simulation with prior distributions of parameters in an image acquisition model that involves image transformations and noise. We compare and combine our proposed aleatoric uncertainty with model uncertainty. Experiments with segmentation of fetal brains and brain tumors from 2D and 3D Magnetic Resonance Images (MRI) showed that 1) the test-time augmentation-based aleatoric uncertainty provides a better uncertainty estimation than calculating the test-time dropout-based model uncertainty alone and helps to reduce overconfident incorrect predictions, and 2) our test-time augmentation outperforms a single-prediction baseline and dropout-based multiple predictions.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/41c9bfb05a7c3eeafe3f749eae1acf6afd873194

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Recent advances and clinical applications of deep learning in medical image analysis (2021, background): “…training (Xue et al., 2018; Zhang et al., 2020b), attention mechanism (Jetley et al., 2018; Anderson et al., 2018; Oktay et al., 2018; Nie et al., 2018; Sinha and Dolz, 2021), and uncertainty estimation (Wang et al., 2019; Yu et al., 2019; Baumgartner et al., 2019; Mehrtash et al., 2020).”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, background): “[33] analyzed epistemic and aleatoric uncertainties for deep CNN-based medical image segmentation problems at both pixel and structure levels.”
A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis (2019, methodology): “Zheng et al. (2015) propose CRF-CNN, a recurrent neural network which has the desirable properties of both CNNs and CRFs. Wang et al. (2019) propose DeepIGeoS, an interactive geodesic framework for medical image segmentation.”
Diffusion Models for Implicit Image Segmentation Ensembles (2021, methodology): “Those methods were also applied on various medical tasks (Wang et al., 2019; Nair et al., 2020; DeVries and Taylor, 2018), including brain tumor segmentation(Sagar, 2020; Jungo and Reyes, 2019; Mehta et al., 2020).”
A survey of uncertainty in deep neural networks (2021, methodology): “uncertainty is estimated in segmentation tasks using approximates of Bayesian inference [1], [2], [4], [152], [261], [262], [263], [264] or test-time data augmentation techniques [249].”
MONAI: An open-source framework for deep learning in healthcare (2022, background): “This is useful if, for example, in test-time augmentation (TTA) [16], augmentation-consistency based domain adaptation, or when the user wishes to save an inferred segmented image in a way that preserves the original geometry and promotes transformation equivariance.”