Brain tumor segmentation with Deep Neural Networks

By Mohammad Havaei, Axel Davy, David Warde-Farley, Antoine Biard, Aaron C. Courville, Yoshua Bengio, C. Pal, Pierre-Marc Jodoin, Hugo Larochelle. 2015

3,119 citations recorded by the source corpus.

&NA; In this paper, we present a fully automatic brain tumor segmentation method based on Deep Neural Networks (DNNs). The proposed networks are tailored to glioblastomas (both low and high grade) pictured in MR images. By their very nature, these tumors can appear anywhere in the brain and have almost any kind of shape, size, and contrast. These reasons motivate our exploration of a machine learning solution that exploits a flexible, high capacity DNN while being extremely efficient. Here, we give a description of different model choices that we've found to be necessary for obtaining competitive performance. We explore in particular different architectures based on Convolutional Neural Networks (CNN), i.e. DNNs specifically adapted to image data. We present a novel CNN architecture which differs from those traditionally used in computer vision. Our CNN exploits both local features as well as more global contextual features simultaneously. Also, different from most traditional uses of CNNs, our networks use a final layer that is a convolutional implementation of a fully connected layer which allows a 40 fold speed up. We also describe a 2‐phase training procedure that allows us to tackle difficulties related to the imbalance of tumor labels. Finally, we explore a cascade architecture in which the output of a basic CNN is treated as an additional source of information for a subsequent CNN. Results reported on the 2013 BRATS test data‐set reveal that our architecture improves over the currently published state‐of‐the‐art while being over 30 times faster. HighlightsA fast and accurate fully automatic method for brain tumor segmentation which is competitive both in terms of accuracy and speed compared to the state of the art.The method is based on deep neural networks (DNN) and learns features that are specific to brain tumor segmentation.We present a new DNN architecture which exploits both local features as well as more global contextual features simultaneously.Using a GPU implementation and a convolutional output layer, the model is an order of magnitude faster than other state of the art methods.Introduces a novel cascaded architecture that allows the system to more accurately model local label dependencies. Graphical abstract Figure. No caption available.

Subfields: Computer Vision and Pattern Recognition, Neurology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Neurology

Citations in the corpus

An overview of deep learning in medical imaging focusing on MRI (2018, background): “example acute ischemic lesion segmentation in DWI [262]; brain tumor segmentation [263]; segmentation of the striatum [264]; segmentation of organs-at-risks in head and neck CT images [265]; and fully automated segmentation of polycystic kidneys [266]; deformable segmentation of the prostate [267]; ”
A systematic study of the class imbalance problem in convolutional neural networks (2017, methodology): “Recently introduced and successfully applied to CNN training for brain tumor segmentation, is two-phase training [28].”
Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images (2016, methodology): “Urban et al. [30] evaluated the use of 3D ﬁlters, although the majority of authors opted for 2D ﬁlters [31]–[35].”
Deep Learning and Medical Diagnosis: A Review of Literature (2018, positive): “[33] DNN MRI Brain tumor segmentation; with this method the whole brain can be segmented in 25 s to 3 min, thus making it a great segmentation tool.”
Learning Deep Representations of Medical Images using Siamese CNNs with Application to Content-Based Image Retrieval (2017, methodology): “…applied to various medical image machine learning tasks, such as image classiﬁcation [Esteva et al., 2017, Gulshan et al., 2016], image segmentation [Havaei et al., 2017, Guo et al., 2017], or content-based image retrieval (CBMIR) [Litjens et al., 2017, Sun et al., 2017, Anavi et al., 2016, Liu…”
Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations (2017, methodology): “The main observed diﬀerence across the diﬀerent loss functions was the robustness to the learning rate, with the WCE and DL 2 being less able to cope with a fast learning rate (10 − 3 ) when using TwoPathCNN while ) 0.06 (0.15) 0.47 (0.32) -4 0.58 (0.20) 0.60 (0.15) 0.61 (0.22) 0.61 (0.18) 0 (0) 0.7”
Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review (2017, methodology): “[24] trained a cascaded architecture, starting by training a two-path CNN.”
Neural Networks and Deep Learning (2018, neutral): “The use of convolutional neural networks for image segmentation is discussed in [180].”
Brain Tumor Detection Based on Deep Learning Approaches and Magnetic Resonance Imaging (2023, methodology): “[61] proposed a 2D and 3D deep CNN architecture for brain tumor segmentation, known as the BraTS network.”
Deep learning based brain tumor segmentation: a survey (2020, background): “Activation feature maps shown in ([34]) (left) and ([90]) (right) respectively.”
HeMIS: Hetero-Modal Image Segmentation (2016, methodology): “To deal with class imbalance, we adopt the patch-wise training procedure mentioned in [8].”
Deep Learning in Medical Image Analysis (2017, background): “…probabilistic graphical models with multiple layers of hidden variables have also been successfully applied to brain disease diagnosis (43, 25, 52, 29), lesion segmentation (53, 45, 32, 54), cell segmentation (33, 55, 34, 56), image parsing (57, 58, 59), and tissue classification (31, 46, 22, 44).”