Self-supervised learning for medical image analysis using image context restoration

By Liang Chen, Paul Bentley, Kensaku Mori, Kazunari Misawa, Michitaka Fujiwara, Daniel Rueckert. Medical Image Anal., 2019

580 citations recorded by the source corpus.

Machine learning, particularly deep learning has boosted medical image analysis over the past years. Training a good model based on deep learning requires large amount of labelled data. However, it is often difficult to obtain a sufficient number of labelled images for training. In many scenarios the dataset in question consists of more unlabelled images than labelled ones. Therefore, boosting the performance of machine learning models by using unlabelled as well as labelled data is an important but challenging problem. Self-supervised learning presents one possible solution to this problem. However, existing self-supervised learning strategies applicable to medical images cannot result in significant performance improvement. Therefore, they often lead to only marginal improvements. In this paper, we propose a novel self-supervised learning strategy based on context restoration in order to better exploit unlabelled images. The context restoration strategy has three major features: 1) it learns semantic image features; 2) these image features are useful for different types of subsequent image analysis tasks; and 3) its implementation is simple. We validate the context restoration strategy in three common problems in medical imaging: classification, localization, and segmentation. For classification, we apply and test it to scan plane detection in fetal 2D ultrasound images; to localise abdominal organs in CT images; and to segment brain tumours in multi-modal MR images. In all three cases, self-supervised learning based on context restoration learns useful semantic features and lead to improved machine learning models for the above tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/d6e78069dbbb35592f013187ef3432286def8f18

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

CutPaste: Self-Supervised Learning for Anomaly Detection and Localization (2021, background): “[11] presents a denoising autoencoder with patch-swap augmentation as noise process.”
Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis (2021, background): “This idea is generalized to inpainting tasks [18, 37, 57] to learn visual representations [3,7,46] by predicting the original image patches.”
COVIDX-Net: A Framework of Deep Learning Classifiers to Diagnose COVID-19 in X-Ray Images (2020, background): “Deep learning techniques showed in the last years promising results to accomplish radiological tasks by automatic analyzing multimodal medical images [27-29].”
Contrastive learning of global and local features for medical image segmentation with limited annotations (2020, methodology): “In recent years, SSL methods [16, 44, 42, 21] have been highly successful for downstream analysis of not only natural images [48, 35, 20], but also medical images [71, 5, 60, 30, 52, 11].”
Big Self-Supervised Models Advance Medical Image Classification (2021, result): “Our results suggest that MICLe constantly improves the performance of skin condition classification over SimCLR [1, 3].”
Self-Supervised Representation Learning: Introduction, advances, and challenges (2021, unknown): “For example, the authors in [75] used unlabeled brain scan images to perform image restoration (an inpainting-like task), improving upon random initialization for fine-tuning several downstream tasks.”
Artificial intelligence and machine learning for medical imaging: A technology review. (2021, unknown): “The literature on self-supervision for medical imaging applications is still scarce [74 – 77], but for instance, a recent work used context restoration as a pretext task [76].”