RingMo: A Remote Sensing Foundation Model With Masked Image Modeling

By Xian Sun, Peijin Wang, Wanxuan Lu, Zicong Zhu, Xiaonan Lu, Qi He, Junxi Li, Xuee Rong, Zhujun Yang, Hao Chang and 5 more. IEEE Transactions on Geoscience and Remote Sensing, 2023

413 citations recorded by the source corpus.

Deep learning approaches have contributed to the rapid development of remote sensing (RS) image interpretation. The most widely used training paradigm is to use ImageNet pretrained models to process RS data for specified tasks. However, there are issues such as domain gap between natural and RS scenes and the poor generalization capacity of RS models. It makes sense to develop a foundation model with general RS feature representation. Since a large amount of unlabeled data is available, the self-supervised method has more development significance than the fully supervised method in RS. However, most of the current self-supervised methods use contrastive learning, whose performance is sensitive to data augmentation, additional information, and selection of positive and negative pairs. In this article, we leverage the benefits of generative self-supervised learning (SSL) for RS images and propose an RS foundation model framework called RingMo, which consists of two parts. First, a large-scale dataset is constructed by collecting two million RS images from satellite and aerial platforms, covering multiple scenes and objects around the world. Second, we propose an RS foundation model training method designed for dense and small objects in complicated RS scenes. We show that the foundation model trained on our dataset with RingMo method achieves state-of-the-art (SOTA) on eight datasets across four downstream tasks, demonstrating the effectiveness of the proposed framework. Through in-depth exploration, we believe it is time for RS researchers to embrace generative SSL and leverage its general representation capabilities to speed up the development of RS applications.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/977a5bf61c63b2c1f83d0c85aaab37c10703db6f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery (2023, methodology): “• The design of SkySense involves three novel technical components: a) A factorized multi-modal spatiotemporal encoder to effectively process multi-modal temporal RSI; b) Multi-Granularity Contrastive Learning that learns features at various levels of granularities to facilitate different tasks; c)…”
SpectralGPT: Spectral Remote Sensing Foundation Model (2023, methodology): “Unlike MAE-based methods that rely on only a few visible image patches to infer the entire image, Sun et al. [27] considered all image patches, whether masked or unmasked, by implementing a MIM strategy in their RS pretraining models.”
EarthGPT: A Universal Multimodal Large Language Model for Multisensor Image Comprehension in Remote Sensing Domain (2024, methodology): “Our goal is to break down the differences between nat-ural images and RS images that lead to the obstacles to the application of MLLMs in the RS domain and propose a universal MLLM specifically customized for RS.”
RemoteCLIP: A Vision Language Foundation Model for Remote Sensing (2023, methodology): “Several recent studies, including SatMAE [12], Scale-MAE [13], ViTAE [14], Billion-scale MAE [15], RingMo [16], and geospatial foundation models (GFM) [17], have employed MIM on large vision transformers (ViT) and large-scale satellite imagery datasets, yielding encouraging results.”
FFCA-YOLO for Small Object Detection in Remote Sensing Images (2024, background): “…research on small object detection has achieved significant growth due to the rapid development of optical remote sensing technology [1], [2], [3], [4], [5], [6] for applications, such as traffic supervision, search and Yin Zhang, Mu Ye, Guiyi Zhu, and Junhua Yan are with the College of…”
Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, methodology): “[96] applied the MAE method to SSFL for a three million unlabeled RSI dataset.”