VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation

By Hao Chen, Qi-Yan Dou, Lequan Yu, P. Heng. 2016

144 citations recorded by the source corpus.

Recently deep residual learning with residual units for training very deep neural networks advanced the state-of-the-art performance on 2D image recognition tasks, e.g., object detection and segmentation. However, how to fully leverage contextual representations for recognition tasks from volumetric data has not been well studied, especially in the field of medical image computing, where a majority of image modalities are in volumetric format. In this paper we explore the deep residual learning on the task of volumetric brain segmentation. There are at least two main contributions in our work. First, we propose a deep voxelwise residual network, referred as VoxResNet, which borrows the spirit of deep residual learning in 2D image recognition tasks, and is extended into a 3D variant for handling volumetric data. Second, an auto-context version of VoxResNet is proposed by seamlessly integrating the low-level image appearance features, implicit shape information and high-level context together for further improving the volumetric segmentation performance. Extensive experiments on the challenging benchmark of brain segmentation from magnetic resonance (MR) images corroborated the efficacy of our proposed method in dealing with volumetric data. We believe this work unravels the potential of 3D deep learning to advance the recognition performance on volumetric image segmentation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

RealPoint3D: Point Cloud Generation from a Single Image with Complex Background (2018, background): “All rights reserved. achieved great success for object classiﬁcation, detection and segmentation (Cicek et al. 2016; Chen et al. 2016; Meagher 1980; Wu et al. 2016; Choy et al. 2016; Mat-urana and Scherer 2015; Tatarchenko, Dosovitskiy, and Brox 2017).”
Vote3Deep: Fast object detection in 3D point clouds using efficient convolutional neural networks (2016, methodology): “A 3D equivalent of residual networks [4] is utilised in [13] for brain image segmentation.”
A survey on deep learning in medical image analysis (2017, background): “Chen et al. (2016d) combined bi-directional LSTM-RNNs with 2D U-net-like-architectures to segment structures in anisotropic 3D electron microscopy images.”
Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes (2016, background): “Some approaches extract features from intermediate layers via some sort of skip connections [38, 8, 36, 7].”
An application of cascaded 3D fully convolutional networks for medical image segmentation (2018, methodology): “…Roth et al. (2016b); Zhou et al. (2016b); Christ et al. (2016); Zhou et al. (2016a), 2D convolutions on orthogonal (2.5D) cross-sections (Roth et al., 2015; Prasoon et al., 2013), and 3D convolutional layers (Milletari et al., 2016; Chen et al., 2016; Dou et al., 2017; Kamnitsas et al., 2017).”
Residual and plain convolutional neural networks for 3D brain MRI classification (2017, background): “There are already publications that show their results for 3D image segmentation like VoxResNet [9], where the authors use ideas of identity connections for ResNets [10].”
Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation (2018, methodology): “This architecture utilizes residual networks and summation of feature maps from different layers [40].”
CE-Net: Context Encoder Network for 2D Medical Image Segmentation (2019, unknown): “[46] proposed a Voxresnet to segment volumetric”
Recurrent residual U-Net for medical image segmentation (2019, methodology): “41 Alternatively, we have proposed two models for semantic segmentation based on the architecture of U-Net in this paper.”
CNN-based Segmentation of Medical Imaging Data (2017, methodology): “Another method seen in [8] and [4] is “deep supervision”.”
Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach (2017, background): “The output of the CNN 2 is the probability Y2 n for each voxel of being part of a WM lesion. 2.3. CNN architecture Although increasingly deep CNN architectures have been proposed in brain MRI lately (Chen et al., 2016; C¸icek et al., 2016; Moeskops et al., 2016b), still those tend to be shallower th”
Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images (2017, background): “Our work is also related to Chen et al. (2016b), which is a 3D extension of residual networks for the segmentation task.”
Tangent Convolutions for Dense Prediction in 3D (2018, background): “Deep networks that operate on voxelized data have been applied to shape classiﬁcation [35, 59, 40], semantic segmentation of indoor scenes [10], and biomedical recordings [9, 8].”
Spatial aggregation of holistically‐nested convolutional neural networks for automated pancreas localization and segmentation☆ (2017, methodology): “Alternatively, there are recent studies on directly using 3D convolutional neural networks for liver, brain segmentation [32], [33] and volumetric vascular boundary detection [34].”
Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks (2017, background): “HighRes3DNet [20] proposes a compact end-to-end 3D CNN structure that maintains high-resolution multi-scale features with dilated convolution and residual connection [29,6].”