A review: Deep learning for medical image segmentation using multi-modality fusion

By Tongxue Zhou, Su Ruan, Stéphane Canu. Array, 2019

606 citations recorded by the source corpus.

Multi-modality is widely used in medical imaging, because it can provide multiinformation about a target (tumor, organ or tissue). Segmentation using multimodality consists of fusing multi-information to improve the segmentation. Recently, deep learning-based approaches have presented the state-of-the-art performance in image classification, segmentation, object detection and tracking tasks. Due to their self-learning and generalization ability over large amounts of data, deep learning recently has also gained great interest in multi-modal medical image segmentation. In this paper, we give an overview of deep learning-based approaches for multi-modal medical image segmentation task. Firstly, we introduce the general principle of deep learning and multi-modal medical image segmentation. Secondly, we present different deep learning network architectures, then analyze their fusion strategies and compare their results. The earlier fusion is commonly used, since it's simple and it focuses on the subsequent segmentation network architecture. However, the later fusion gives more attention on fusion strategy to learn the complex relationship between different modalities. In general, compared to the earlier fusion, the later fusion can give more accurate result if the fusion method is effective enough. We also discuss some common problems in medical image segmentation. Finally, we summarize and provide some perspectives on the future research.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Neurology

View Paper: https://www.semanticscholar.org/paper/d5dec7c20409e1034073ae0353b16b3914392669

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Neurology

Citations in the corpus

TransMed: Transformers Advance Multi-Modal Medical Image Classification (2021, methodology): “The existing computer vision models based on transformer mainly deal with 2D natural images, such as ImageNet [6] and other large-scale datasets.”
Deep semantic segmentation of natural and medical images: a review (2019, background): “Compared to the other review papers on deep learning-based semantic image segmentation for natural and medical images [35, 41, 47, 52, 71, 140, 179], we make the following contributions: • We provide comprehensive coverage of research contributions in the ﬁeld of natural and medical image semantic…”
Medical Image Segmentation Using Deep Learning: A Survey (2020, background): “More relevant literatures can be found in the review on multi-modal fusion for medical image segmentation using deep learning [152].”
Deep Multimodal Data Fusion (2024, background): “Compared to the existing surveys [2, 4, 45, 46, 121, 128, 227, 235, 243] that usually focus on one single task (such as multimodal object recognition) with one specific combination of two modalities (such as RGB+depth data), this survey owns a broader scope covering various modalities and their…”