Deep Learning-based Image Segmentation on Multimodal Medical Imaging

By Zhe Guo, Xiang Li, Heng Huang, Ning Guo, Quanzheng Li. IEEE Transactions on Radiation and Plasma Medical Sciences, 2019

366 citations recorded by the source corpus.

Multimodality medical imaging techniques have been increasingly applied in clinical practice and research studies. Corresponding multimodal image analysis and ensemble learning schemes have seen rapid growth and bring unique value to medical applications. Motivated by the recent success of applying deep learning methods to medical image processing, we first propose an algorithmic architecture for supervised multimodal image analysis with cross-modality fusion at the feature learning level, classifier level, and decision-making level. We then design and implement an image segmentation system based on deep convolutional neural networks to contour the lesions of soft tissue sarcomas using multimodal images, including those from magnetic resonance imaging, computed tomography, and positron emission tomography. The network trained with multimodal images shows superior performance compared to networks trained with single-modal images. For the task of tumor segmentation, performing image fusion within the network (i.e., fusing at convolutional or fully connected layers) is generally better than fusing images at the network output (i.e., voting). This paper provides empirical guidance for the design and application of multimodal image analysis.

Subfields: Artificial Intelligence, Neurology, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/30c17f09515c8e407a3cf599b6faaedd0cb2ca8f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Neurology

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Deep learning and medical image processing for coronavirus (COVID-19) pandemic: A survey (2020, positive): “The authors in Guo, Li, Huang, Guo, and Li (2019) propose a supervised Artificial Neural Network (ANN) technique with the combination of cross-modality, which is applied in all the levels of ANN.”
A review: Deep learning for medical image segmentation using multi-modality fusion (2019, background): “Therefore, applying multi-modal images can reduce the information uncertainty and improve clinical diagnosis and segmentation accuracy [22].”
Medical image analysis using deep learning algorithms (2023, methodology): “Case Study: A study conducted by Guo et al. (8) at Massachusetts General Hospital utilized DL techniques to automate the segmentation of brain tumors from MRI scans.”