Spectral–Spatial Feature Tokenization Transformer for Hyperspectral Image Classification

By Guangrui Zhao, Yuhui Zheng, Zebin Wu. IEEE Transactions on Geoscience and Remote Sensing, 2022

791 citations recorded by the source corpus.

In hyperspectral image (HSI) classification, each pixel sample is assigned to a land-cover category. In the recent past, convolutional neural network (CNN)-based HSI classification methods have greatly improved performance due to their superior ability to represent features. However, these methods have limited ability to obtain deep semantic features, and as the layer’s number increases, computational costs rise significantly. The transformer framework can represent high-level semantic features well. In this article, a spectral–spatial feature tokenization transformer (SSFTT) method is proposed to capture spectral–spatial features and high-level semantic features. First, a spectral–spatial feature extraction module is built to extract low-level features. This module is composed of a 3-D convolution layer and a 2-D convolution layer, which are used to extract the shallow spectral and spatial features. Second, a Gaussian weighted feature tokenizer is introduced for features transformation. Third, the transformed features are input into the transformer encoder module for feature representation and learning. Finally, a linear layer is used to identify the first learnable token to obtain the sample label. Using three standard datasets, experimental analysis confirms that the computation time is less than other deep learning methods and the performance of the classification outperforms several current state-of-the-art methods. The code of this work is available at https://github.com/zgr6010/HSI_SSFTT for the sake of reproducibility.

Subfields: Atmospheric Science, Media Technology

View Paper: https://www.semanticscholar.org/paper/d532a98f64f243d08c2532e5ccc9f7c6b3669fe8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Media Technology

Citations in the corpus

Multimodal Fusion Transformer for Remote Sensing Image Classification (2022, methodology): “Inspired by the success of transformers in computer vision [49], several new transformer models have been developed in the last few years [50].”
MambaHSI: Spatial–Spectral Mamba for Hyperspectral Image Classification (2025, background): “DL-based be roughly divided into GCN-based [11], [13], CNN-based [15], [31], and Transformer-based [32], [33] methods.”
Extended Vision Transformer (ExViT) for Land Use and Land Cover Classification: A Multimodal Deep Learning Framework (2023, background): “[37] mainly focused on the tokenization process and developed a 2-D and 3-D convolution-integrated transformer that can express feature semantics more adequately.”
Spectral–Spatial Morphological Attention Transformer for Hyperspectral Image Classification (2023, methodology): “To further improve the classification performance of HSIs, Sun et al. [48] introduced spatial and spectral tokenization of feature representations in the encoder, which helps to extract local spatial information and establish long-range relations between neighboring sequences.”