Contrastive learning of global and local features for medical image segmentation with limited annotations

By Krishna Chaitanya, Ertunç Erdil, Neerav Karani, Ender Konukoglu. Neural Information Processing Systems, 2020

697 citations recorded by the source corpus.

A key requirement for the success of supervised deep learning is a large labeled dataset - a condition that is difficult to meet in medical image analysis. Self-supervised learning (SSL) can help in this regard by providing a strategy to pre-train a neural network with unlabeled data, followed by fine-tuning for a downstream task with limited annotations. Contrastive learning, a particular variant of SSL, is a powerful technique for learning image-level representations. In this work, we propose strategies for extending the contrastive learning framework for segmentation of volumetric medical images in the semi-supervised setting with limited annotations, by leveraging domain-specific and problem-specific cues. Specifically, we propose (1) novel contrasting strategies that leverage structural similarity across volumetric medical images (domain-specific cue) and (2) a local version of the contrastive loss to learn distinctive representations of local regions that are useful for per-pixel segmentation (problem-specific cue). We carry out an extensive evaluation on three Magnetic Resonance Imaging (MRI) datasets. In the limited annotation setting, the proposed method yields substantial improvements compared to other self-supervision and semi-supervised learning techniques. When combined with a simple data augmentation technique, the proposed method reaches within 8% of benchmark performance using only two labeled MRI volumes for training, corresponding to only 4% (for ACDC) of the training data used to train the benchmark.

Subfields: Artificial Intelligence, Pediatrics, Perinatology and Child Health, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/72d926dfe60e0de18f3cd034de51bca22ceda3f2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Pediatrics, Perinatology and Child Health

Mingxia Liu

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

Exploring Cross-Image Pixel Contrast for Semantic Segmentation (2021, background): “Though a few concurrent works also address contrastive learning in dense image prediction [75, 7, 69], the ideas are significantly different.”
MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretation (2021, background): “In medical imaging, previous work has found performance improvement when applying contrastive learning to chest X-ray interpretation (Sowrirajan et al., 2020; Sriram et al., 2021; Azizi et al., 2021), dermatology classiﬁcation (Azizi et al. (2021)) and MRI segmentation (Chaitanya et al., 2020).”
SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation (2021, background): “Contrastive learning, a sub-area of self-supervised learning, has recently been noted as a promising direction since it has shown great promise in learning useful representations with limited human supervision [26], [27], [28], [24].”
Recent advances and clinical applications of deep learning in medical image analysis (2021, methodology): “Chaitanya et al. (2020) claimed although the contrastive loss in (Chen et al., 2020a) was suitable for learning image-level (global) feature representations, it did not guarantee learning distinctive local representations that are important for per-pixel segmentation.”
Federated learning for medical image analysis: A survey (2023, methodology): “Contrastive learning (Chaitanya et al., 2020; He et al., 2020; Misra and Maaten, 2020) is a self-supervised method that can learn useful representations”
Mutual consistency learning for semi-supervised medical image segmentation (2021, background): “The contrastive learning-based models (Chaitanya et al., 2020) can perform self-supervised training to mitigate the over-fitting of deep models.”
SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining (2021, neutral): “Al-ternatively, contrastive learning methods have been proposed to leverage unsupervised data for improved generalisation ability [8, 71].”
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends (2023, methodology): “This natural scenario makes SSL a compelling approach, which has been effectively employed for various tasks like medical image segmentation [240] and 3D medical image analysis [241].”
Joint Learning of Localized Representations from Medical Images and Reports (2021, background): “Therefore, there is a number of recent approaches that use region-level contrast[87,86,83,6,61,52], i.”