Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
By Konstantinos Kamnitsas, Christian Ledig, Virginia F. J. Newcombe, Joanna P. Simpson, Andrew D. Kane, David K. Menon, Daniel Rueckert, Ben Glocker. 2016
3,360 citations recorded by the source corpus.
HIGHLIGHTSAn efficient 11‐layers deep, multi‐scale, 3D CNN architecture.A novel training strategy that significantly boosts performance.The first employment of a 3D fully connected CRF for post‐processing.State‐of‐the‐art performance on three challenging lesion segmentation tasks.New insights into the automatically learned intermediate representations. ABSTRACT We propose a dual pathway, 11‐layers deep, three‐dimensional Convolutional Neural Network for the challenging task of brain lesion segmentation. The devised architecture is the result of an in‐depth analysis of the limitations of current networks proposed for similar applications. To overcome the computational burden of processing 3D medical scans, we have devised an efficient and effective dense training scheme which joins the processing of adjacent image patches into one pass through the network while automatically adapting to the inherent class imbalance present in the data. Further, we analyze the development of deeper, thus more discriminative 3D CNNs. In order to incorporate both local and larger contextual information, we employ a dual pathway architecture that processes the input images at multiple scales simultaneously. For post‐processing of the network's soft segmentation, we use a 3D fully connected Conditional Random Field which effectively removes false positives. Our pipeline is extensively evaluated on three challenging tasks of lesion segmentation in multi‐channel MRI patient data with traumatic brain injuries, brain tumours, and ischemic stroke. We improve on the state‐of‐the‐art for all three applications, with top ranking performance on the public benchmarks BRATS 2015 and ISLES 2015. Our method is computationally efficient, which allows its adoption in a variety of research and clinical settings. The source code of our implementation is made publicly available.
Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition, Medical Imaging
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luc Van Gool
- Sergey I. Nikolenko
- Mohit Bansal
- Yoshua Bengio
- Quoc V. Le
- Percy Liang
- Sergey Levine
- Dacheng Tao
In Biomedical Engineering
- Daniel Rueckert
- Andriy Myronenko
- P. Heng
- Adrian V. Dalca
- Tom Kamiel Magda Vercauteren
- Sébastien Ourselin
- Guha Balakrishnan
- John V. Guttag
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Jitendra Malik
- Xiaogang Wang
- Trevor Darrell
- Andrea Vedaldi
- Andrew Zisserman
In Medical Imaging
- Daniel Rueckert
- Andriy Myronenko
- Sergey I. Nikolenko
- Sébastien Ourselin
- Ben Glocker
- Konstantinos Kamnitsas
- Tom Kamiel Magda Vercauteren
- Francesco Ciompi
Citations in the corpus
- V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation (2016, neutral): “Post-processing approaches such as connected components analysis normally yield no improvement and therefore, more recent works, propose to use the network predictions in combination with Markov random fields [6], voting strategies [9] or more traditional approaches such as level-sets [2].”
- GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks (2018, background): “DeepMedic [9] is a popular general purpose segmentation algorithm which has been shown to perform well in many applications, and was therefore chosen as a third network to explore.”
- A survey on deep learning in medical image analysis (2017, methodology): “These models, also referred to as dual pathway architectures (Kamnitsas et al., 2017), have two main applications at the time of writing: (1) multi-scale image analysis and (2) 2.5D classification; both relevant for medical image processing tasks.”
- Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach (2017, methodology): “However, compared with the latter, the lower variation in MRI tissues permits the use of smaller networks, which tends to reduce overfitting, specially in small training sets.”
- Three-dimensional convolutional neural network (3D-CNN) for heterogeneous material homogenization (2020, background): “It is worth noticing that the proposed 3D-CNN approach is more advantageous for heterogeneous materials with multiple constituents and extremely complex microstructure since it has demonstrated extraordinary ability in handling high-dimensional inputs [51–54].”
- FastSurfer - A fast and accurate deep learning based neuroimaging pipeline (2019, neutral): “eral deep learning based approaches have been proposed for specific tasks, such as tumor segmentation [22, 23, 24, 25, 26, 27], brain lesion segmentation [28, 29, 30, 31, 32], MR image reconstruction [33, 34, 35, 36, 37, 38] or prediction of brain related diseases and their progression [39, 40, 41, ”
- State-of-the-art review on deep learning in medical imaging. (2019, methodology): “This motivated the development of DL-based architecture for brain lesion segmentation (47).”
- Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields (2016, neutral): “[20,8,23,21,19,12].”
- DeepMedic for Brain Tumor Segmentation (2016, methodology): “DeepMedic is the 11-layers deep, multi-scale 3D CNN we presented in [1] for brain lesion segmentation.”
- Deep Learning for Medical Image Processing: Overview, Challenges and Future (2017, methodology): “Alban [8] de-noised the angiograph images of EyePACS and then applied CNNs for detection of DR.”
- Unsupervised domain adaptation in brain lesion segmentation with adversarial networks (2016, methodology): “The obtained performance is similar to what was reported in [10], although on a different database.”
- Learning Data Augmentation for Brain Tumor Segmentation with Coarse-to-Fine Generative Adversarial Networks (2018, methodology): “[3] reported a running time of 3 minutes using a 3GB GPU to segment one case, while 6-12 minutes was reported by Zhao [12].”