HSI-BERT: Hyperspectral Image Classification Using the Bidirectional Encoder Representation From Transformers

By Ji He, Lina Zhao, Hongwei Yang, Mengmeng Zhang, Wei Li. IEEE Transactions on Geoscience and Remote Sensing, 2020

354 citations recorded by the source corpus.

Deep learning methods have been widely used in hyperspectral image classification and have achieved state-of-the-art performance. Nonetheless, the existing deep learning methods are restricted by a limited receptive field, inflexibility, and difficult generalization problems in hyperspectral image classification. To solve these problems, we propose HSI-BERT, where BERT stands for bidirectional encoder representations from transformers and HSI stands for hyperspectral imagery. The proposed HSI-BERT has a global receptive field that captures the global dependence among pixels regardless of their spatial distance. HSI-BERT is very flexible and enables the flexible and dynamic input regions. Furthermore, HSI-BERT has good generalization ability because the jointly trained HSI-BERT can be generalized from regions with different shapes without retraining. HSI-BERT is primarily built on a multihead self-attention (MHSA) mechanism in an MHSA layer. Moreover, several attentions are learned by different heads, and each head of the MHSA layer encodes the semantic context-aware representation to obtain discriminative features. Because all head-encoded features are merged, the resulting features exhibit spatial–spectral information that is essential for accurate pixel-level classification. Quantitative and qualitative results demonstrate that HSI-BERT outperforms any other CNN-based model in terms of both classification accuracy and computational time and achieves state-of-the-art performance on three widely used hyperspectral image data sets.

Subfields: Atmospheric Science, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/3994a5d35795ec2a7c1e95930044c21a23398f16

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Self-Supervised Pretraining of Transformers for Satellite Image Time Series Classification (2020, methodology): “Transformer was ﬁrst introduced in NLP as an efﬁcient alternative to RNNs [28], which has been introduced to some remote sensing tasks, such as hyperspectral image classiﬁcation [53], image captioning [54], and SITS classiﬁcation [29].”
Vision Transformers for Remote Sensing Image Classification (2021, methodology): “For instance, He et al. [46] proposed a model derived from the bidirectional encoder representations called BERT [47] that was used in the natural language processing ﬁeld to the context of hyperspectral images.”
Spectral–Spatial Morphological Attention Transformer for Hyperspectral Image Classification (2023, background): “He et al. [46] proposed a bidirectional encoder representation for a transformer that incorporates flexible and dynamic input regions for pixel-based classification of HSIs.”
Remote Sensing Image Change Detection With Transformers (2021, background): “He et al. [47] applied a variant of the transformer (BERT [51]) to capture global dependencies among pixels in hyperspectral image classiﬁcation.”
Transformers in Remote Sensing: A Survey (2022, methodology): “Bert [114] and other existing CNN-based methods on the Pavia dataset.”