Capsule Networks for Hyperspectral Image Classification

By Mercedes Eugenia Paoletti, Juan Mario Haut, Rubén Fernández-Beltran, Javier Plaza, Antonio J. Plaza, Jun Yu Li, Filiberto Pla. IEEE Transactions on Geoscience and Remote Sensing, 2019

311 citations recorded by the source corpus.

Convolutional neural networks (CNNs) have recently exhibited an excellent performance in hyperspectral image classification tasks. However, the straightforward CNN-based network architecture still finds obstacles when effectively exploiting the relationships between hyperspectral imaging (HSI) features in the spectral–spatial domain, which is a key factor to deal with the high level of complexity present in remotely sensed HSI data. Despite the fact that deeper architectures try to mitigate these limitations, they also find challenges with the convergence of the network parameters, which eventually limit the classification performance under highly demanding scenarios. In this paper, we propose a new CNN architecture based on spectral–spatial capsule networks in order to achieve a highly accurate classification of HSIs while significantly reducing the network design complexity. Specifically, based on Hinton’s capsule networks, we develop a CNN model extension that redefines the concept of capsule units to become spectral–spatial units specialized in classifying remotely sensed HSI data. The proposed model is composed by several building blocks, called spectral–spatial capsules, which are able to learn HSI spectral–spatial features considering their corresponding spatial positions in the scene, their associated spectral signatures, and also their possible transformations. Our experiments, conducted using five well-known HSI data sets and several state-of-the-art classification methods, reveal that our HSI classification approach based on spectral–spatial capsules is able to provide competitive advantages in terms of both classification accuracy and computational time.

Subfields: Biomedical Engineering, Media Technology

View Paper: https://www.semanticscholar.org/paper/e3ab2c2277f77ffd49bbd1392cb3b5b576a70f48

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Media Technology

Citations in the corpus

Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review (2019, unknown): “Examples are found in [209–212].”
Deep Learning for Hyperspectral Image Classification: An Overview (2019, background): “capsules to extract spectral–spatial features of HSIs [103].”
A Survey of Deep Active Learning (2020, background): “Thanks to the publication of a large number of existing annotation datasets [21, 120], in recent years, DL has made breakthroughs in various fields including machine translation [5, 18, 217, 231], speech recognition [152, 159, 169, 189], and image classification [89, 144, 158, 239].”
Attention-Based Adaptive Spectral–Spatial Kernel ResNet for Hyperspectral Image Classification (2020, methodology): “[46] developed a deep pyramidal residual network (ResNet) (DPyResNet) and a CNN architecture based on Hinton’s CapsNets [47] for fast and accurate HSI classification.”
HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image Classification (2019, background): “capsule networks to learn the hyperspectral features [30], whereas Fang et al.”
SpectralFormer: Rethinking Hyperspectral Image Classification With Transformers (2021, background): “In recent years, many well-recognized backbone networks have been widely and successfully applied in the HS image classification task [23], such as autoencoders (AEs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), capsule networks (Ca”
Classification of Hyperspectral Image Based on Double-Branch Dual-Attention Mechanism Network (2019, unknown): “In [40], spectral-spatial capsule networks (CapsNets) were designed to weaken the complexity of the network and enhance the accuracy of the classification.”
Spatial-Spectral Transformer for Hyperspectral Image Classification (2021, methodology): “After that, several deep learning models such as the deep belief network [20], CNN [21,22], recurrent neural network [23,24], generative adversarial network [25,26], and capsule network [27,28] were investigated for HSI classification and obtained good classification performance.”
HSI-BERT: Hyperspectral Image Classification Using the Bidirectional Encoder Representation From Transformers (2020, background): “The classification tasks of HSI span a broad range of applications, such as ecological science [3], [5], precision agriculture [6]–[8], mineral exploration [9], and medicinel [10].”
Spectral–Spatial Feature Tokenization Transformer for Hyperspectral Image Classification (2022, background): “In the meantime, other networks based on generative adversarial networks (GANs) [51], [52], capsule networks [53], and graph convolutional networks (GCNs) [54] have also been introduced for HSI classiﬁcation.”