A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection

By Zhenwei Shi. Remote Sensing, 2020

1,845 citations recorded by the source corpus.

Remote sensing image change detection (CD) is done to identify desired significant changes between bitemporal images. Given two co-registered images taken at different times, the illumination variations and misregistration errors overwhelm the real object changes. Exploring the relationships among different spatial–temporal pixels may improve the performances of CD methods. In our work, we propose a novel Siamese-based spatial–temporal attention neural network. In contrast to previous methods that separately encode the bitemporal images without referring to any useful spatial–temporal dependency, we design a CD self-attention mechanism to model the spatial–temporal relationships. We integrate a new CD self-attention module in the procedure of feature extraction. Our self-attention module calculates the attention weights between any two pixels at different times and positions and uses them to generate more discriminative features. Considering that the object may have different scales, we partition the image into multi-scale subregions and introduce the self-attention in each subregion. In this way, we could capture spatial–temporal dependencies at various scales, thereby generating better representations to accommodate objects of various sizes. We also introduce a CD dataset LEVIR-CD, which is two orders of magnitude larger than other public datasets of this field. LEVIR-CD consists of a large set of bitemporal Google Earth images, with 637 image pairs (1024 × 1024) and over 31 k independently labeled change instances. Our proposed attention module improves the F1-score of our baseline model from 83.9 to 87.3 with acceptable computational overhead. Experimental results on a public remote sensing image CD dataset show our method outperforms several other state-of-the-art methods.

Subfields: Atmospheric Science, Ecology, Media Technology

View Paper: https://www.semanticscholar.org/paper/2778c4820d364ab8d0e28166de7a0045b1e8dc2f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Ecology

In Media Technology

Citations in the corpus

Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation (2022, methodology): “Table 17: Comparison with state-of-the-art methods on the LEVIR-CD dataset [9].”
SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery (2023, methodology): “1) LEVIR-CD [9].”
A Deeply Supervised Attention Metric-Based Network and an Open Aerial Image Dataset for Remote Sensing Change Detection (2021, methodology): “…been speciﬁcally proposed to monitor changes in buildings, which is one of the most common and concerned types in CD, including the Wuhan University (WHU) Building CD [36], AIST Building Change Detection (ABCD) dataset [37], and the Learning, Vision and Remote Sensing Laboratory (LEVIR)-CD [29].”
Change Detection on Remote Sensing Images Using Dual-Branch Multilevel Intertemporal Network (2023, background): “1) LEarning, VIsion, and Remote sensing (LEVIR-CD) [50] [36] includes 20 000 orthographic aerial pairs with a spatial size of 256 × 256 and a spatial resolution of 0.5 m, which are gathered in Hong Kong.”
RingMo: A Remote Sensing Foundation Model With Masked Image Modeling (2023, methodology): “We use UCM [34], AID [35], NWPU-RESISC45 [36] for scene recognition experiments, FAIR1M [4] and DIOR [5] for object detection experiments, iSAID [37] and ISPRS Postdam 2 for semantic segmentation experiments, and LEVIR-CD [38] for change detection experiments.”
ChangeMamba: Remote Sensing Change Detection With Spatiotemporal State Space Model (2024, background): “3) LEVIR-CD+ [56]: This advanced version of the LEVIR-CD dataset introduces a comprehensive resource for BCD research.”
SNUNet-CD: A Densely Connected Siamese Network for Change Detection of VHR Images (2021, methodology): “In [10], the pyramid model is used for extracting multiscale features; in [9] and [12], deep super-vision is used to enhance the representation and discrimination capabilities of shallow features; and attention mechanisms are used to reﬁne features and obtain better feature representations, such as…”
RemoteCLIP: A Vision Language Foundation Model for Remote Sensing (2023, methodology): “We combined six remote sensing datasets with object detection annotation, including DOTA [75], DIOR [76], HRRSD [77], RSOD [78], LEVIR [79], and HRSC [80].”
Changer: Feature Interaction is What You Need for Change Detection (2022, background): “Some work has tried implicit alignment, such as the use of global attention in [5].”
SwinSUNet: Pure Transformer Network for Remote Sensing Image Change Detection (2022, methodology): “We compared SwinSUNet with several excellent methods, such as CDNet [48], FC-EF [20], FC-Siam-conc [20], FC-Siam-diff [20], DASNet [22] and STANet [21].”
An Empirical Study of Remote Sensing Pretraining (2022, methodology): “• LEVIR: This dataset is collected using the GE API on 20 different regions in Texas, USA from 2002 to 2018.”
RS-Mamba for Large Remote Sensing Image Dense Prediction (2024, methodology): “…along the horizontal axis, and VMamba designed SS2D module which introduces selective scanning across both the horizontal and vertical directions, allowing every part of the image to globally attend in both forward and backward directions along a specific direction, as illustrated in Fig.”
Transformers in Remote Sensing: A Survey (2022, methodology): “RSP [7] Classification, Segmentation, Detection MillionAID [33], Potsdam [70], iSAID [80], HRSC2016 [81], DOTA [49], CCD [82], LEVIR [61] Overall classification accuracy, mAP, F1 score Investigates pre-training transformers on a large-scale remote sensing dataset.”