Global and Local Contrastive Self-Supervised Learning for Semantic Segmentation of HR Remote Sensing Images

By Haifeng Li, Yi Li, Guo Zhang, Ruoyun Liu, Haozhe Huang, Qing Zhu, Chao Tao. 2021

153 citations recorded by the source corpus.

Recently, supervised deep learning has achieved a great success in remote sensing image (RSI) semantic segmentation. However, supervised learning for semantic segmentation requires a large number of labeled samples, which is difficult to obtain in the field of remote sensing. A new learning paradigm, self-supervised learning (SSL), can be used to solve such problems by pretraining a general model with a large number of unlabeled images and then fine-tuning it on a downstream task with very few labeled samples. Contrastive learning is a typical method of SSL that can learn general invariant features. However, most existing contrastive learning methods are designed for classification tasks to obtain an image-level representation, which may be suboptimal for semantic segmentation tasks requiring pixel-level discrimination. Therefore, we propose a global style and local matching contrastive learning network (GLCNet) for RSI semantic segmentation. Specifically, first, the global style contrastive learning module is used to better learn an image-level representation, as we consider that style features can better represent the overall image features. Next, the local features matching the contrastive learning module is designed to learn the representations of local regions, which is beneficial for semantic segmentation. We evaluate four RSI semantic segmentation datasets, and the experimental results show that our method mostly outperforms the state-of-the-art self-supervised methods and the ImageNet pretraining method. Specifically, with 1% annotation from the original dataset, our approach improves Kappa by 6% on the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam dataset relative to the existing baseline. Moreover, our method outperforms supervised learning methods when there are some differences between the datasets of upstream tasks and downstream tasks. Our study promotes the development of SSL in the field of RSI semantic segmentation. Since SSL could directly learn the essential characteristics of data from unlabeled data, which is easy to obtain in the remote sensing field, this may be of great significance for tasks such as global mapping. The source code is available at https://github.com/GeoX-Lab/G-RSIM.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Chao Tao
2. Haifeng Li
3. Ji Qi
4. Mingning Guo
5. Qing Zhu
6. Valerio Marsocci
7. Simone Scardapane
8. Laura Palagi

In Computer Vision and Pattern Recognition

1. Haifeng Li
2. Chao Tao
3. Qing Zhu
4. Mingning Guo
5. Ji Qi
6. Valerio Marsocci
7. Simone Scardapane
8. Laura Palagi

In Media Technology

1. Simone Scardapane
2. Valerio Marsocci
3. Ji Qi
4. Haifeng Li
5. Qing Zhu
6. Mingning Guo
7. Chao Tao

Citations in the corpus

• Continual Barlow Twins: Continual Self-Supervised Learning for Remote Sensing Semantic Segmentation (2022, background): “In [35], a global style and local matching contrastive learning network is proposed.”
• FALSE: False Negative Samples Aware Contrastive Learning for Semantic Segmentation of High-Resolution Remote Sensing Image (2022, unknown): “The experiments were selected from the public RSI semantic segmentation dataset ISPRS Potsdam [24], competition dataset CVPR DGLC [25], and Xiangtan dataset [10] from the Gaofen-2 satellite covering Xiangtan, China.”
• Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, background): “[167] constructed a global style and local matching contrastive learning network (GLCNet) to extend instance-level contrast learning to super-pixel-level contrastive learning.”