Med3D: Transfer Learning for 3D Medical Image Analysis

By Sihong Chen, Kai Ma, Yefeng Zheng. arXiv.org, 2019

681 citations recorded by the source corpus.

The performance on deep learning is significantly affected by volume of training data. Models pre-trained from massive dataset such as ImageNet become a powerful weapon for speeding up training convergence and improving accuracy. Similarly, models based on large dataset are important for the development of deep learning in 3D medical images. However, it is extremely challenging to build a sufficiently large dataset due to difficulty of data acquisition and annotation in 3D medical imaging. We aggregate the dataset from several medical challenges to build 3DSeg-8 dataset with diverse modalities, target organs, and pathologies. To extract general medical three-dimension (3D) features, we design a heterogeneous 3D network called Med3D to co-train multi-domain 3DSeg-8 so as to make a series of pre-trained models. We transfer Med3D pre-trained models to lung segmentation in LIDC dataset, pulmonary nodule classification in LIDC dataset and liver segmentation on LiTS challenge. Experiments show that the Med3D can accelerate the training convergence speed of target 3D medical tasks 2 times compared with model pre-trained on Kinetics dataset, and 10 times compared with training from scratch as well as improve accuracy ranging from 3% to 20%. Transferring our Med3D model on state-the-of-art DenseASPP segmentation network, in case of single model, we achieve 94.6\% Dice coefficient which approaches the result of top-ranged algorithms on the LiTS challenge.

Subfields: Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/5bcda431e0b615e094562bf038f1ef4df1865088

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

LViT: Language Meets Vision Transformer in Medical Image Segmentation (2022, methodology): “A summary of existing solutions is shown in Figure 1(a): (1) one shared encoder followed by two separate decoders [1]; (2) two separate encoders followed by one shared decode [2]; (3) two separate encoders followed”
Deep Learning for Cardiac Image Segmentation: A Review (2019, background): “Transfer learning can help to decrease the training time and achieve lower generalization error (85).”
Brain Imaging Generation with Latent Diffusion Models (2022, methodology): “The FID was calculated using an approach similar to [31], where features were extracted using a pretrained Med3D [5].”
On the Challenges and Perspectives of Foundation Models for Medical Image Analysis (2023, methodology): “…imaging modalities such as radiology images (including X-ray, CT, MR, and Ultrasound) Ghesu et al. [2022], 3D images (stacks of 2D CT and MR images) Chen et al. [2019], or a particular medical imaging modality which includes X-ray Tiu et al. [2022], CT Huang et al. [2023b], Wang et al. [2023a],…”
MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer (2023, background): “Diffusion models have recently demonstrated significant potential in various segmentation tasks, including medical images (Armato III et al. 2011; Caron et al. 2021; Cao et al. 2022; Chen, Ma, and Zheng 2019).”
Big Self-Supervised Models Advance Medical Image Classification (2021, background): “For example, [7, 20, 26, 13] report performance improvements when pretraining on labeled data in the same domain.”
Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis (2021, background): “Prior studies have demonstrated the effectiveness of supervised pre-training in medical imaging for different applications [9, 39].”
A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, background): “[56] combine multiple data sets from several medical challenges with diverse modalities, target organs, and pathologies and learn one 3-D network that provides an effective pretrained model for 3-D medical image analysis tasks.”