Spatial aggregation of holistically‐nested convolutional neural networks for automated pancreas localization and segmentation☆

By Holger R. Roth, Le Lu, Nathan S. Lay, Adam P. Harrison, Amal A. Farag, Andrew Sohn, Ronald M. Summers. Medical Image Anal., 2017

300 citations recorded by the source corpus.

&NA; Accurate and automatic organ segmentation from 3D radiological scans is an important yet challenging problem for medical image analysis. Specifically, as a small, soft, and flexible abdominal organ, the pancreas demonstrates very high inter‐patient anatomical variability in both its shape and volume. This inhibits traditional automated segmentation methods from achieving high accuracies, especially compared to the performance obtained for other organs, such as the liver, heart or kidneys. To fill this gap, we present an automated system from 3D computed tomography (CT) volumes that is based on a two‐stage cascaded approach—pancreas localization and pancreas segmentation. For the first step, we localize the pancreas from the entire 3D CT scan, providing a reliable bounding box for the more refined segmentation step. We introduce a fully deep‐learning approach, based on an efficient application of holistically‐nested convolutional networks (HNNs) on the three orthogonal axial, sagittal, and coronal views. The resulting HNN per‐pixel probability maps are then fused using pooling to reliably produce a 3D bounding box of the pancreas that maximizes the recall. We show that our introduced localizer compares favorably to both a conventional non‐deep‐learning method and a recent hybrid approach based on spatial aggregation of superpixels using random forest classification. The second, segmentation, phase operates within the computed bounding box and integrates semantic mid‐level cues of deeply‐learned organ interior and boundary maps, obtained by two additional and separate realizations of HNNs. By integrating these two mid‐level cues, our method is capable of generating boundary‐preserving pixel‐wise class label maps that result in the final pancreas segmentation. Quantitative evaluation is performed on a publicly available dataset of 82 patient CT scans using 4‐fold cross‐validation (CV). We achieve a (mean ± std. dev.) Dice similarity coefficient (DSC) of 81.27 ± 6.27% in validation, which significantly outperforms both a previous state‐of‐the art method and a preliminary version of this work that report DSCs of 71.80 ± 10.70% and 78.01 ± 8.20%, respectively, using the same dataset.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/3d44a1a97b6c768ae3080adf717326167457b0ad

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Attention U-Net: Learning Where to Look for the Pancreas (2018, methodology): “Implementation Details: In contrast to the state-of-the-art CNN segmentation frameworks [4, 26], we propose a 3D-model to capture sufﬁcient semantic context.”
DRINet for Medical Image Segmentation (2018, methodology): “Furthermore, deep learning approaches are much faster than conventional methods [4], [39], [40].”
An application of cascaded 3D fully convolutional networks for medical image segmentation (2018, methodology): “Most methods are based on variants of FCNs (Long et al., 2015) that either employ 2D convolutional layers in a slice-by-slice fashion Roth et al. (2016b); Zhou et al. (2016b); Christ et al. (2016); Zhou et al. (2016a), 2D convolutions on orthogonal (2.5D) cross-sections (Roth et al., 2015; Prasoon e”
Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation (2020, background): “Moreover, [51,69] illustrates the effectiveness of multi-view training on 2D slices, even by simply averaging multi-planar results, indicating complementary latent information exists in the biases of 2D networks.”
A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, methodology): “Several groups have used data sets, such as TCIA CT pancreas, to improve pancreas segmentation with Dice coefficients reaching the mid 80 percentile [193]–[196].”
Reducing the Hausdorff Distance in Medical Image Segmentation With Convolutional Neural Networks (2019, background): “pancreas, identify boundary cues, and aggregate segmentation cues [54].”
Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images (2018, unknown): “The results in Table 3 and 4 show improved performance compared to concatenated multi-model CNN approaches (Cai et al., 2017; Roth et al., 2018; Zhou et al., 2017) due to additional training data and richer semantic information (e.g. spleen labels).”