Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis

By Zongwei Zhou, V. Sodha, Mahfuzur R. Siddiquee, Ruibin Feng, Nima Tajbakhsh, Michael B. Gotway, Jianming Liang. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019

406 citations recorded by the source corpus.

Transfer learning from natural image to medical image has established as one of the most practical paradigms in deep learning for medical image analysis. However, to fit this paradigm, 3D imaging tasks in the most prominent imaging modalities (e.g., CT and MRI) have to be reformulated and solved in 2D, losing rich 3D anatomical information and inevitably compromising the performance. To overcome this limitation, we have built a set of models, called Generic Autodidactic Models, nicknamed Models Genesis, because they are created ex nihilo (with no manual labeling), self-taught (learned by self-supervision), and generic (served as source models for generating application-specific target models). Our extensive experiments demonstrate that our Models Genesis significantly outperform learning from scratch in all five target 3D applications covering both segmentation and classification. More importantly, learning a model from scratch simply in 3D may not necessarily yield performance better than transfer learning from ImageNet in 2D, but our Models Genesis consistently top any 2D approaches including fine-tuning the models pre-trained from ImageNet as well as fine-tuning the 2D versions of our Models Genesis, confirming the importance of 3D anatomical information and significance of our Models Genesis for 3D medical imaging. This performance is attributed to our unified self-supervised learning framework, built on a simple yet powerful observation: the sophisticated yet recurrent anatomy in medical images can serve as strong supervision signals for deep models to learn common anatomical representation automatically via self-supervision. As open science, all pre-trained Models Genesis are available at https://github.com/MrGiovanni/ModelsGenesis.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/75dd6e09e2a44c363bb50e463c812fd3393cbf1c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises (2020, methodology): “The models’ genesis method [61] uses a proxy task of recovering the original image using a distorted image as input.”
SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation (2021, background): “Semi-Supervised Medical Image Segmentation In recent years, substantial efforts [15], [16], [17], [41], [42], [14], [20], [12], [19], [43], [44], [45] have been devoted to incorporating unlabeled data to improve network performance due to limited annotations.”
Annotation-efficient deep learning for automatic medical image segmentation (2020, background): “Methods such as data augmentation 41,46 , transfer learning 42,43 , semi-supervised learning 44,47 , and self-supervised learning 48,49 have been extensively investigated to handle cases with limited training annotations or no target domain annotations.”