RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation Model

By Keyan Chen, Chenyang Liu, Hao Chen, Haotian Zhang, Wenyuan Li, Zhengxia Zou, Zhenwei Shi. IEEE Transactions on Geoscience and Remote Sensing, 2023

474 citations recorded by the source corpus.

Leveraging the extensive training data from SA-1B, the segment anything model (SAM) demonstrates remarkable generalization and zero-shot capabilities. However, as a category-agnostic instance segmentation method, SAM heavily relies on prior manual guidance, including points, boxes, and coarse-grained masks. Furthermore, its performance in remote sensing image segmentation tasks remains largely unexplored and unproven. In this article, we aim to develop an automated instance segmentation approach for remote sensing images based on the foundational SAM model and incorporating semantic category information. Drawing inspiration from prompt learning, we propose a method to learn the generation of appropriate prompts for SAM. This enables SAM to produce semantically discernible segmentation results for remote sensing images, a concept that we have termed RSPrompter. We also propose several ongoing derivatives for instance segmentation tasks, drawing on recent advancements within the SAM community, and compare their performance with RSPrompter. Extensive experimental results, derived from the WHU building dataset, the NWPU VHR-10 dataset, and the SAR Ship Detection Dataset (SSDD) dataset, validate the effectiveness of our proposed method. The code for our method is publicly available at https://kychen.me/RSPrompter.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/37defa9aa690c033f9c4d1510df32ae5e996f716

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Foundation Models Defining a New Era in Vision: A Survey and Outlook (2025, unknown): “Further, such a generic visual prompt-based segmentation model can be adapted for speciﬁc downstream tasks such as medical image segmentation [12], [13], video object segmentation [14], robotics [15], and remote sensing [16].”
SAM 2: Segment Anything in Images and Videos (2024, background): “…its adoption in a wide range of applications, such as medical imaging (Ma et al., 2024; Deng et al., 2023; Mazurowski et al., 2023; Wu et al., 2023a), remote sensing (Chen et al., 2024; Ren et al., 2024), motion segmentation (Xie et al., 2024), and camouflaged object detection (Tang et al., 2023).”