Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation

By Tanya Nair, Doina Precup, Douglas L. Arnold, Tal Arbel. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018

510 citations recorded by the source corpus.

Deep learning networks have recently been shown to outperform other segmentation methods on various public, medical-image challenge datasets, particularly on metrics focused on large pathologies. For diseases such as Multiple Sclerosis (MS), however, monitoring all the focal lesions visible on MRI sequences, even very small ones, is essential for disease staging, prognosis, and evaluating treatment efficacy. Small lesion segmentation presents significant challenges to popular deep learning models. This, coupled with their deterministic predictions, hinders their clinical adoption. Uncertainty estimates for these predictions would permit subsequent revision by clinicians. We present the first exploration of multiple uncertainty estimates based on Monte Carlo (MC) dropout (Gal and Ghahramani, 2016) in the context of deep networks for lesion detection and segmentation in medical images. Specifically, we develop a 3D MS lesion segmentation CNN, augmented to provide four different voxel-based uncertainty measures based on MC dropout. We train the network on a proprietary, large-scale, multi-site, multi-scanner, clinical MS dataset, and compute lesion-wise uncertainties by accumulating evidence from voxel-wise uncertainties within detected lesions. We analyze the performance of voxel-based segmentation and lesion-level detection by choosing operating points based on the uncertainty. Uncertainty filtering improves both voxel and lesion-wise TPR and FDR on remaining, certain predictions compared to sigmoid-based TPR/FDR curves. Small lesions and lesion-boundaries are the most uncertain regions, which is consistent with human-rater variability.

Subfields: Artificial Intelligence, Biophysics, Media Technology

View Paper: https://www.semanticscholar.org/paper/e5614f59dc657d7add799373712dd5baee68f330

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biophysics

In Media Technology

Citations in the corpus

On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation (2020, methodology): “More concretely, deep models are now applied to diverse ﬁelds such as physics [ABCG + 15, RWR + 18, HRH18], biology [AIP + 15], healthcare [LAA + 17, NPAA18, LKN + 18], or autonomous driving [KG17, MKG18] to name a few.”
Probabilistic Object Detection: Definition and Evaluation (2018, methodology): “[31] provide four different voxel-based uncertainty measures for their 3D lesion segmentation system to enable a more complete revision by clinicians.”
A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, background): “Recently, there are emerging works that quantify uncertainty in DL methods for medical image segmentation [95]–[97], lesion detection [98], chest X-ray disease classification [99], and diabetic retinopathy grading [100], [101].”
A survey of uncertainty in deep neural networks (2021, methodology): “uncertainty is estimated in segmentation tasks using approximates of Bayesian inference [1], [2], [4], [152], [261], [262], [263], [264] or test-time data augmentation techniques [249].”
On the Interpretability of Artificial Intelligence in Radiology: Challenges and Opportunities. (2020, methodology): “Uncertainty estimation has been proposed to assess voxelwise confidence levels of a DL model trained to segment structures on an image and to use these estimates to drive user corrections (27) or eliminate unconfi-dent areas from further quantification tasks (28,29).”
Deep Learning for Medical Anomaly Detection – A Survey (2020, methodology): “Furthermore, Bayesian uncertainty estimation has been applied for estimating the quality of medical image segmentation [135], [136], [137], and sclerosis lesion detection [138].”
Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks (2018, unknown): “Nair et al. [15] and Roy et al. [16] used Monte Carlo samples generated by test-time dropout for structure/lesion-wise uncertainty estimation.”
A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, methodology): “[35] conducted MC dropout to estimate four types of uncertainties, including variance of MC samples, predictive entropy, and Mutual Information (MI), in a 3D CNN to segment lesion from MRI sequences.”