Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data

By Oscar Mañas, Alexandre Lacoste, Xavier Giró-i-Nieto, David Vázquez, Pau Rodr'iguez L'opez. 2021

383 citations recorded by the source corpus.

Remote sensing and automatic earth monitoring are key to solve global-scale challenges such as disaster prevention, land use monitoring, or tackling climate change. Although there exist vast amounts of remote sensing data, most of it remains unlabeled and thus inaccessible for supervised learning algorithms. Transfer learning approaches can reduce the data requirements of deep learning algorithms. However, most of these methods are pre-trained on ImageNet and their generalization to remote sensing imagery is not guaranteed due to the domain gap. In this work, we propose Seasonal Contrast (SeCo), an effective pipeline to leverage unlabeled data for in-domain pre-training of remote sensing representations. The SeCo pipeline is composed of two parts. First, a principled procedure to gather large-scale, unlabeled and uncurated remote sensing datasets containing images from multiple Earth locations at different timestamps. Second, a self-supervised algorithm that takes advantage of time and position invariance to learn transferable representations for remote sensing applications. We empirically show that models trained with SeCo achieve better performance than their ImageNet pre-trained counterparts and state-of-the-art self-supervised learning methods on multiple downstream tasks. The datasets and models in SeCo will be made public to facilitate transfer learning and enable rapid progress in remote sensing applications.1

Subfields: Artificial Intelligence, Global and Planetary Change, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Global and Planetary Change

In Media Technology

Citations in the corpus

SpectralGPT: Spectral Remote Sensing Foundation Model (2023, methodology): “ResNet50 [37] ImageNet-1k 96.72 SeCo [38] SeCo 97.23 ViT [22] Random Init.”
Advancing Plain Vision Transformer Toward Remote Sensing Foundation Model (2022, background): “A few self-supervised learning (SSL) methods [15], [16] have been proposed, which however only target CNNs and have not been proven effective for large-scale vision transformer mod-”
SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery (2022, methodology): “We follow [31, 56] for the train/val splits on EuroSAT.”
RingMo: A Remote Sensing Foundation Model With Masked Image Modeling (2023, background): “Furthermore, some researchers [24], [25], [29], [30] introduce additional information speciﬁc to RS (geographic information, time-series data, audio data, etc.) through careful search or matching and encourage models to learn similarities with additional information in addition to data augmentation.”
Geographical Knowledge-Driven Representation Learning for Remote Sensing Images (2021, background): “Recent researches have included geographical information in contrastive learning in order to achieve more efficient representation learning [24, 25].”
Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, methodology): “[69] proposed the seasonal contrast framework (SeCo) to learn the temporal-invariant features.”
RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation Model (2023, methodology): “We have conducted experiments employing the following methods: ResNet50 pre-trained on ImageNet [65], ViT-B pretrained with MAE [81], SatMAE [95], Scale-MAE [96], SeCo [97], and CACo [98].”
Changer: Feature Interaction is What You Need for Change Detection (2022, methodology): “In general, the metric-based method transforms the two input images into a feature space whose feature representation becomes more consistent [27, 32].”
Satlas: A Large-Scale, Multi-Task Dataset for Remote Sensing Image Understanding (2022, methodology): “Several methods [5, 10, 38, 46, 47] incorporate temporal augmentations into a contrastive learning framework, where images of the same location captured at different times are encouraged to have closer representations than images of different locations.”
Self-Supervised Learning in Remote Sensing: A review (2022, methodology): “Early progress is shown in Figure 21, where seasonal information is used for data augmentation [201].”
An Empirical Study of Remote Sensing Pretraining (2022, methodology): “On the first image of school scene, the RSPViTAEv2-S simultaneously focuses on the playground and the surroundings, surpassing the SeCo-ResNet-50.”
EarthGPT: A Universal Multimodal Large Language Model for Multisensor Image Comprehension in Remote Sensing Domain (2024, methodology): “Many specialist models such as SeCo [37], CSDS [38], MGSNet [39], T-CNN [40], and PSGAN [41], have made progress on scene classification.”