Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding

By Vladan Stojnić, Vladimir Risojevic. 2021

129 citations recorded by the source corpus.

In recent years self-supervised learning has emerged as a promising candidate for unsupervised representation learning. In the visual domain its applications are mostly studied in the context of images of natural scenes. However, its applicability is especially interesting in specific areas, like remote sensing and medicine, where it is hard to obtain huge amounts of labeled data. In this work, we conduct an extensive analysis of the applicability of self-supervised learning in remote sensing image classification. We analyze the influence of the number and domain of images used for self-supervised pre-training on the performance on downstream tasks. We show that, for the downstream task of remote sensing image classification, using self-supervised pre-training on remote sensing images can give better results than using supervised pre-training on images of natural scenes. Besides, we also show that self-supervised pre-training can be easily extended to multispectral images producing even better results on our downstream tasks.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Chao Tao
2. Valerio Marsocci
3. Jiwen Lu
4. Yi Wang
5. Chenlin Meng
6. F. Liu
7. Ji Qi
8. Stefano Ermon

Citations in the corpus

• Unsupervised Single-Scene Semantic Segmentation for Earth Observation (2022, background): “Weakly supervised [48], [51], [52], unsupervised [53], and self-supervised learning [54], [55] have also been used in many remote sensing applications, e.g., cloud detection [52], change detection [53], and scene classification [54].”
• SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery (2022, background): “[42, 43, 44] apply off-the-shelf contrastive learning algorithms to satellite images.”
• RingMo: A Remote Sensing Foundation Model With Masked Image Modeling (2023, background): “While Stojnic and Risojevic [47] leverage CMC on different RS datasets.”
• Continual Contrastive Learning for Cross-Dataset Scene Classification (2022, background): “[25] analyzed the effect of sample size and domain of scene images for training, and their work demonstrated that results of pretrained models by contrastive learning outperform others on scene classification.”
• RemoteCLIP: A Vision Language Foundation Model for Remote Sensing (2023, methodology): “Stojnic and Risojevic [37] applied contrastive multiview coding (CMC) to learn schematic representations, which were subsequently adapted for downstream classification tasks.”
• Self-Supervised Learning in Remote Sensing: A review (2022, methodology): “Stojnic and Risojevic [198]: contrastive multiview coding (CMC)”
• Global and Local Contrastive Self-Supervised Learning for Semantic Segmentation of HR Remote Sensing Images (2021, unknown): “remote sensing and proved its feasibility [40]–[43].”
• Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, methodology): “Category-Level Contrastive Learning Signal: Instance-level contrastive learning signals have been widely used for RSI feature learning [172], [173], [174], [175], [176], [177], [178],”