Deep Unsupervised Embedding for Remotely Sensed Images Based on Spatially Augmented Momentum Contrast

By Jian Kang, Rubén Fernández-Beltran, Puhong Duan, Sicong Liu, Antonio J. Plaza. 2020

108 citations recorded by the source corpus.

Convolutional neural networks (CNNs) have achieved great success when characterizing remote sensing (RS) images. However, the lack of sufficient annotated data (together with the high complexity of the RS image domain) often makes supervised and transfer learning schemes limited from an operational perspective. Despite the fact that unsupervised methods can potentially relieve these limitations, they are frequently unable to effectively exploit relevant prior knowledge about the RS domain, which may eventually constrain their final performance. In order to address these challenges, this article presents a new unsupervised deep metric learning model, called spatially augmented momentum contrast (SauMoCo), which has been specially designed to characterize unlabeled RS scenes. Based on the first law of geography, the proposed approach defines spatial augmentation criteria to uncover semantic relationships among land cover tiles. Then, a queue of deep embeddings is constructed to enhance the semantic variety of RS tiles within the considered contrastive learning process, where an auxiliary CNN model serves as an updating mechanism. Our experimental comparison, including different state-of-the-art techniques and benchmark RS image archives, reveals that the proposed approach obtains remarkable performance gains when characterizing unlabeled scenes since it is able to substantially enhance the discrimination ability among complex land cover categories. The source codes of this article will be made available to the RS community for reproducible research.

Subfields: Computer Vision and Pattern Recognition, Environmental Engineering, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Simone Scardapane
3. Valerio Marsocci
4. Bo Du
5. Jing Zhang
6. Di Wang
7. Guisong Xia
8. Zhenwei Shi

In Environmental Engineering

1. Dacheng Tao
2. Simone Scardapane
3. Xiao Xiang Zhu
4. Xingjian Shi
5. Lichao Mou
6. Guisong Xia

In Media Technology

1. Simone Scardapane
2. Valerio Marsocci
3. Zhenwei Shi
4. Bo Du
5. Guisong Xia
6. Dacheng Tao
7. Jing Zhang
8. Di Wang

Citations in the corpus

• Continual Barlow Twins: Continual Self-Supervised Learning for Remote Sensing Semantic Segmentation (2022, background): “Other contrastive strategies are proposed by [20] and [21].”
• RemoteCLIP: A Vision Language Foundation Model for Remote Sensing (2023, background): “Kang et al. [32], Jung et al. [33], and Jean et al. [34] utilized spatial neighbors as augmented data.”
• An Empirical Study of Remote Sensing Pretraining (2022, background): “Moreover, the channel properties [53] and spatial variance [51] are also explored in some approaches.”
• Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, background): “beled RSIs for scene classification [164], [165], [166].”
• Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding (2021, methodology): “Most of these applications either used small amounts, up to 50,000 images, of unlabeled training data [31, 30, 35] or tested learned representations on a small number of datasets [1, 19, 38].”
• Self-Supervised Learning in Remote Sensing: A review (2022, background): “, considering the spatial relationship of neighboring and distant patches to sample positive/negative pairs [177, 196] as shown in Fig.”
• Towards Geospatial Foundation Models via Continual Pretraining (2023, methodology): “[24] and [21] also utilize a MoCo-inspired objective, but specify a cropping procedure to generate positives and negatives within”
• Geographical Knowledge-Driven Representation Learning for Remote Sensing Images (2021, methodology): “Self-supervised representation learning has also been used to improve the networks pre-training for remote sensing images [22, 23].”