Vision Transformers for Remote Sensing Image Classification

By Yakoub Bazi, Laila Bashmal, Mohamad Mahmoud Al Rahhal, Reham Al Dayil, Naif Al Ajlan. Remote Sensing, 2021

575 citations recorded by the source corpus.

In this paper, we propose a remote-sensing scene-classification method based on vision transformers. These types of networks, which are now recognized as state-of-the-art models in natural language processing, do not rely on convolution layers as in standard convolutional neural networks (CNNs). Instead, they use multihead attention mechanisms as the main building block to derive long-range contextual relation between pixels in images. In a first step, the images under analysis are divided into patches, then converted to sequence by flattening and embedding. To keep information about the position, embedding position is added to these patches. Then, the resulting sequence is fed to several multihead attention layers for generating the final representation. At the classification stage, the first token sequence is fed to a softmax classification layer. To boost the classification performance, we explore several data augmentation strategies to generate additional data for training. Moreover, we show experimentally that we can compress the network by pruning half of the layers while keeping competing classification accuracies. Experimental results conducted on different remote-sensing image datasets demonstrate the promising capability of the model compared to state-of-the-art methods. Specifically, Vision Transformer obtains an average classification accuracy of 98.49%, 95.86%, 95.56% and 93.83% on Merced, AID, Optimal31 and NWPU datasets, respectively. While the compressed version obtained by removing half of the multihead attention layers yields 97.90%, 94.27%, 95.30% and 93.05%, respectively.

Subfields: Atmospheric Science, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/f8b5340b221ac2ac7c2067f28032c00636e6fc14

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Remote Sensing Image Change Detection With Transformers (2021, background): “…of transformer models on NLP/CV tasks has intrigued the remote sensing community to study their applications in remote sensing tasks, such as image time-series classiﬁcation [45, 46], hyperspectral image classiﬁcation [47], scene classiﬁcation [48], and remote sensing image captioning [49, 50].”
UNetFormer: A UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery (2021, unknown): “Driven by this, many researchers in the remote sensing field have applied the Transformer for remote sensing image scene classification (Bazi et al., 2021; Deng et al., 2021), hyperspectral image classification (Hong et al., 2021) (He et al., 2021; Zhong et al., 2021), object detection (Li et al.,…”
Transformers in Remote Sensing: A Survey (2022, methodology): “This has started a new wave of promising research in remote sensing with different approaches utilizing either ImageNet pre-training [4]–[6] or performing remote sensing pre-training [7] with vision transformers.”