Remote Sensing Image Change Detection With Transformers

By Zipeng Qi, Zhenwei Shi. IEEE Transactions on Geoscience and Remote Sensing, 2021

1,602 citations recorded by the source corpus.

Modern change detection (CD) has achieved remarkable success by the powerful discriminative ability of deep convolutions. However, high-resolution remote sensing CD remains challenging due to the complexity of objects in the scene. Objects with the same semantic concept may show distinct spectral characteristics at different times and spatial locations. Most recent CD pipelines using pure convolutions are still struggling to relate long-range concepts in space-time. Nonlocal self-attention approaches show promising performance via modeling dense relationships among pixels, yet are computationally inefficient. Here, we propose a bitemporal image transformer (BIT) to efficiently and effectively model contexts within the spatial-temporal domain. Our intuition is that the high-level concepts of the change of interest can be represented by a few visual words, that is, semantic tokens. To achieve this, we express the bitemporal image into a few tokens and use a transformer encoder to model contexts in the compact token-based space-time. The learned context-rich tokens are then fed back to the pixel-space for refining the original features via a transformer decoder. We incorporate BIT in a deep feature differencing-based CD framework. Extensive experiments on three CD datasets demonstrate the effectiveness and efficiency of the proposed method. Notably, our BIT-based model significantly outperforms the purely convolutional baseline using only three times lower computational costs and model parameters. Based on a naive backbone (ResNet18) without sophisticated structures (e.g., feature pyramid network (FPN) and UNet), our model surpasses several state-of-the-art CD methods, including better than four recent attention-based methods in terms of efficiency and accuracy. Our code is available at https://github.com/justchenhao/BIT_CD.

Subfields: Artificial Intelligence, Ecology, Media Technology

View Paper: https://www.semanticscholar.org/paper/561377c8e51bb864a1a689d830fab2f0881ae78e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Ecology

In Media Technology

Citations in the corpus

A Transformer-Based Siamese Network for Change Detection (2022, methodology): “We also show better performance in terms of IoU, F1 score, and overall accuracy than recent ConvNet-based (FC-EF, FC-Siam-DI, and FC-Siam-Conc), attention-based (DTCDSCN, STANet, and IFNet), and ConvNet+Transformer-based (BIT) methods.”
ChangeMamba: Remote Sensing Change Detection With Spatiotemporal State Space Model (2024, methodology): “On the Transformer front, we compare against ChangeFormer (versions 1 to 4) [22], BIT (with ResNet-18 to ResNet-101 as the backbone for feature extraction) [20], TransUNetCD [25], and SwinSUNet [24].”
Changer: Feature Interaction is What You Need for Change Detection (2022, methodology): “BiT [18] is a hybrid model of CNN and transformer.”
RS-Mamba for Large Remote Sensing Image Dense Prediction (2024, methodology): “In recent years, deep learning models based on transformers [7] have been widely applied to VHR remote sensing dense prediction tasks [8], [9], [10], [11].”
Change Detection on Remote Sensing Images Using Dual-Branch Multilevel Intertemporal Network (2023, methodology): “…network (IFNet) [35], dual-task constrained deep siamese convolutional network (DTCDSCN) [40], densely connected siamese network (SNUNet) [37], BIT [31], edge-guided recurrent CNN (EGR-CNN) [17], deep multiscale siamese network with parallel convolutional structure and self-attention (MSPSNet)…”
UNetFormer: A UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery (2021, neutral): “…hyperspectral image classification (Hong et al., 2021) (He et al., 2021; Zhong et al., 2021), object detection (Li et al., 2022a), change detection (Chen et al., 2021a), building and road extraction (Chen et al., 2021c) (Sun et al., 2022), and especially semantic segmentation (Li et al., 2020b)…”
Transformers in Remote Sensing: A Survey (2022, background): “CD-Trans [54] Change Detection WHU [60], LEVIR [61], DSIFN [96] F1 score Introduces a bi-temporal image transformer designed to model the spatio-temporal contextual information.”
An Empirical Study of Remote Sensing Pretraining (2022, methodology): “(d) and (n) are the results of the IMP-ResNet-50 bas d BIT, while (e) and (o), (f) and (p , (f) and (o), (g) and (q), ( ) and (r), (i) and (s), (g) and (t) are the results from the SeCo-ResNet-50, RSP-ResNet-50, IMP-Swin-T, RSP-Swin-T, IMP-ViTAEv2-S, and RSP-ViTAEv2-S backbones, respectively. the re”