Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis

By Rongjian Li, Wenlu Zhang, Heung-Il Suk, Li Wang, Jiang Li, Dinggang Shen, Shuiwang Ji. 2014

470 citations recorded by the source corpus.

Combining multi-modality brain data for disease diagnosis commonly leads to improved performance. A challenge in using multi-modality data is that the data are commonly incomplete; namely, some modality might be missing for some subjects. In this work, we proposed a deep learning based framework for estimating multi-modality imaging data. Our method takes the form of convolutional neural networks, where the input and output are two volumetric modalities. The network contains a large number of trainable parameters that capture the relationship between input and output modalities. When trained on subjects with all modalities, the network can estimate the output modality given the input modality. We evaluated our method on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, where the input and output modalities are MRI and PET images, respectively. Results showed that our method significantly outperformed prior methods.

Subfields: Computer Vision and Pattern Recognition, Neurology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Neurology

Citations in the corpus

Deep learning in neural networks: An overview (2014, neutral): “and Alzheimer’s disease neuroimaging (Li et al., 2014).”
A survey on deep learning in medical image analysis (2017, methodology): “Avendi et al. (2016) MRI CNN Left ventricle segmentation; AE used to initialize filters because training data set was small”
Deep Learning in Medical Image Analysis (2017, methodology): “To this end, research groups have devised various strategies, such as (i) to take image patches either 2D or 3D as input (25, 35, 36, 37, 38, 39, 40, 41), rather than the full-sized images, to reduce the input dimensionality, thus the number of model parameters; (ii) to expand their dataset by…”
Medical Image Synthesis with Context-Aware Generative Adversarial Networks (2016, methodology): “In the following paragraphs, we will describe the framework that we use in the MRI-to-CT prediction.”
Introduction to artificial intelligence in medicine (2019, background): “…similar databased patients, a diagnosis could be suggested as well as treatment protocols, outcomes and prognosis, all based on evidence-based medicine such as RCTs (randomized controlled studies), best practice guidelines, electronic medical records (EMRs), public health records etc. [11,28 – 31].”
Globally and locally consistent image completion (2017, methodology): “Similar approaches have also been applied to MRI and PET images for completing missing data [Li et al. 2014].”
Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images (2017, background): “Some studies started to employ 3D CNN to cope with detection and segmentation problems in medical volumetric data (Li et al. 2014; Brosch et al. 2016; Kamnitsas et al. 2016; Chen et al. 2016a).”
Medical Image Synthesis with Deep Convolutional Adversarial Networks (2018, methodology): “[32] applied a similar deep learningmodel to estimate themissing PET image from theMRI data of the same subject.”
Multimodal MR Synthesis via Modality-Invariant Latent Representation (2017, methodology): “More recently, neural networks have been applied to MR synthesis and segmentation, and like many of the sparse coding based methods, often they approach the problem as a patch based regression [21].”
Deep learning in radiology: an overview of the concepts and a survey of the state of the art with focus on MRI (2018, methodology): “In another application to aid a classiﬁcation framework for Alzheimer’s disease diagnosis with missing PET scans, PET patterns were predicted from MRI using CNN [120].”
Multi-Modality Cascaded Convolutional Neural Networks for Alzheimer’s Disease Diagnosis (2018, methodology): “…voxel-wise features of multi-modality images including the GM density map of MRI and the intensity values of PET are combined with a sparse regression classifier, and a deep learning based framework was proposed for estimating missing PET imaging data for multimodal classification (Li et al. 2014).”
ResViT: Residual Vision Transformers for Multimodal Medical Image Synthesis (2021, background): “Earlier studies in this domain have proposed local networks based on patch-level processing [16], [61],”
Hi-Net: Hybrid-Fusion Network for Multi-Modal MR Image Synthesis (2020, methodology): “[33] used a deep learning model to estimate the missing positron emission tomography (PET) data from the corresponding MRI data.”