3D fully convolutional networks for subcortical segmentation in MRI: A large‐scale study

By Jose Dolz, Christian Desrosiers, Ismail Ben Ayed. NeuroImage, 2016

388 citations recorded by the source corpus.

ABSTRACT This study investigates a 3D and fully convolutional neural network (CNN) for subcortical brain structure segmentation in MRI. 3D CNN architectures have been generally avoided due to their computational and memory requirements during inference. We address the problem via small kernels, allowing deeper architectures. We further model both local and global context by embedding intermediate‐layer outputs in the final prediction, which encourages consistency between features extracted at different scales and embeds fine‐grained information directly in the segmentation process. Our model is efficiently trained end‐to‐end on a graphics processing unit (GPU), in a single stage, exploiting the dense inference capabilities of fully CNNs. We performed comprehensive experiments over two publicly available datasets. First, we demonstrate a state‐of‐the‐art performance on the ISBR dataset. Then, we report a large‐scale multi‐site evaluation over 1112 unregistered subject datasets acquired from 17 different sites (ABIDE dataset), with ages ranging from 7 to 64 years, showing that our method is robust to various acquisition protocols, demographics and clinical factors. Our method yielded segmentations that are highly consistent with a standard atlas‐based approach, while running in a fraction of the time needed by atlas‐based methods and avoiding registration/normalization steps. This makes it convenient for massive multi‐site neuroanatomical imaging studies. To the best of our knowledge, our work is the first to study subcortical structure segmentation on such large‐scale and heterogeneous data.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/090dc6adb27cd5e4048746f0ee2a2fd087746530

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review (2017, methodology): “According to the literature, this intensity normalisation can be driven in two ways: (i) histogram matching [121, 122, 123, 15, 22] and (ii) normalise data to achieve zero mean and unit variance [22, 24, 27, 28, 124].”
HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation (2018, methodology): “While sub-volumes of size 27⇥27⇥27 are considered for training, we used 35⇥35⇥35 sub-volumes during inference, as in [15], [36].”
RA-UNet: A Hybrid Deep Attention-Aware Network to Extract Liver and Tumor in CT Scans (2018, background): “Dolz et al. investigated a 3D FCN for subcortical brain structure segmentation in MRI images [21].”
A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, methodology): “Recent research has focused on important tasks, such as DL-based brain extraction [123], cortical and subcortical segmentation [124]–[127], and tumor and lesion segmentation [128], [129].”
Multi-Scale Self-Guided Attention for Medical Image Segmentation (2019, background): “Many extensions of these networks have been proposed to solve pixel-wise segmentation problems in a wide range of applications [30], [31], [32], [33], [34], [35], [36], [37], [38], [39].”
3D Deep Learning on Medical Images: A Review (2020, methodology): “In Reference [62], a 3D CNN was implemented for subcortical brain structure segmentation in MRI and this study was based on the effect of the size of the kernels in a network.”
FastSurfer - A fast and accurate deep learning based neuroimaging pipeline (2019, neutral): “…or segmentation of a smaller number of brain (sub-)structures (Zhang et al., 2015; Akkus et al., 2017; Milletari et al., 2017; Fedorov et al., 2017; Dolz et al., 2018; Thyreau et al., 2018; Chen et al., 2018; Nogovitsyn et al., 2019; Li et al., 2019; Sun et al., 2019; Ito et al., 2019) full brain…”
IterNet: Retinal Image Segmentation Utilizing Structural Redundancy in Vessel Networks (2019, background): “Image segmentation: Currently, most state-of-the-art models [8, 9, 10] for semantic segmentation stem from a fully-convolutional design, which is first introduced by the fully convolutional network (FCN) [11].”
3D conditional generative adversarial networks for high-quality PET image estimation at low dose (2018, background): “Convolutional neural networks (CNN) have drawn a tremendous amount of attention in machine learning and medical image analysis areas (Kamnitsas et al., 2017; Kleesiek et al., 2016; Valverde et al., 2017; Dolz et al., 2017; Kawahara et al., 2017).”
Boundary loss for highly unbalanced segmentation (2018, background): “Recent years have witnessed a substantial growth in the number of deep learning methods for medical image segmentation ( Litjens et al., 2017; Shen et al., 2017; Dolz et al., 2018; Ker et al., 2018 ).”