HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image Classification

By Swalpa Kumar Roy, Gopal Krishna, Shiv Ram Dubey, Bidyut Baran Chaudhuri. IEEE Geoscience and Remote Sensing Letters, 2019

1,500 citations recorded by the source corpus.

Hyperspectral image (HSI) classification is widely used for the analysis of remotely sensed images. Hyperspectral imagery includes varying bands of images. Convolutional neural network (CNN) is one of the most frequently used deep learning-based methods for visual data processing. The use of CNN for HSI classification is also visible in recent works. These approaches are mostly based on 2-D CNN. On the other hand, the HSI classification performance is highly dependent on both spatial and spectral information. Very few methods have used the 3-D-CNN because of increased computational complexity. This letter proposes a hybrid spectral CNN (HybridSN) for HSI classification. In general, the HybridSN is a spectral–spatial 3-D-CNN followed by spatial 2-D-CNN. The 3-D-CNN facilitates the joint spatial–spectral feature representation from a stack of spectral bands. The 2-D-CNN on top of the 3-D-CNN further learns more abstract-level spatial representation. Moreover, the use of hybrid CNNs reduces the complexity of the model compared to the use of 3-D-CNN alone. To test the performance of this hybrid approach, very rigorous HSI classification experiments are performed over Indian Pines, University of Pavia, and Salinas Scene remote sensing data sets. The results are compared with the state-of-the-art hand-crafted as well as end-to-end deep learning-based methods. A very satisfactory performance is obtained using the proposed HybridSN for HSI classification. The source code can be found at https://github.com/gokriznastic/HybridSN.

Subfields: Atmospheric Science, Ecology, Media Technology

View Paper: https://www.semanticscholar.org/paper/25b9153473ae156113f6fd2dd9a28c2d2f104a8f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Ecology

In Media Technology

Citations in the corpus

Multimodal Fusion Transformer for Remote Sensing Image Classification (2022, background): “Then, the extracted features are sent to a spectral classiﬁer [1], [23]–[26].”
CNN-Enhanced Graph Convolutional Network With Pixel- and Superpixel-Level Feature Fusion for Hyperspectral Image Classification (2020, methodology): “…method: they are the deep CNN (DCNN) [25], the TCCNN [26], the MCCNN [31], the spectral–spatial residual network (SSRN) [33], the fast dense spectral–spatial convolutional (FDSSC) network [34], the hybrid spectral network (HybridSN) [27], and the double-branch dual-attention network (DBDA) [38].”
Attention-Based Adaptive Spectral–Spatial Kernel ResNet for Hyperspectral Image Classification (2020, methodology): “In our prior work [34], we proposed HybridSN using the sequential”
Spectral–Spatial Morphological Attention Transformer for Hyperspectral Image Classification (2023, background): “Then, spectral classifiers are fed with such features for classification purposes [2], [7], [19], [20], [21], [22].”
Spectral–Spatial Feature Tokenization Transformer for Hyperspectral Image Classification (2022, methodology): “Roy et al. [41] combined the characteristics of 3-D-and 2-D-CNNs and proposed a hierarchical network structure.”
Hyperspectral Image Classification—Traditional to Deep Models: A Survey for Future Prospects (2021, methodology): “A hybrid 3D − 2D-CNN architecture was presented by [166] in which 3D-CNN is first used to extract joint spectral-spatial features and then 2D-CNN is further used to obtain more abstract spatial contextual features.”