Spectral-Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network

By Ying Li, Haokui Zhang, Qiang Shen. Remote Sensing, 2017

1,079 citations recorded by the source corpus.

Recent research has shown that using spectral–spatial information can considerably improve the performance of hyperspectral image (HSI) classification. HSI data is typically presented in the format of 3D cubes. Thus, 3D spatial filtering naturally offers a simple and effective method for simultaneously extracting the spectral–spatial features within such images. In this paper, a 3D convolutional neural network (3D-CNN) framework is proposed for accurate HSI classification. The proposed method views the HSI cube data altogether without relying on any preprocessing or post-processing, extracting the deep spectral–spatial-combined features effectively. In addition, it requires fewer parameters than other deep learning-based methods. Thus, the model is lighter, less likely to over-fit, and easier to train. For comparison and validation, we test the proposed method along with three other deep learning-based HSI classification methods—namely, stacked autoencoder (SAE), deep brief network (DBN), and 2D-CNN-based methods—on three real-world HSI datasets captured by different sensors. Experimental results demonstrate that our 3D-CNN-based method outperforms these state-of-the-art methods and sets a new record.

Subfields: Atmospheric Science, Media Technology

View Paper: https://www.semanticscholar.org/paper/ba14fa8493a1919903c6b9ab047a86448f7e2559

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

In Media Technology

Citations in the corpus

Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series (2018, positive): “The main successful application of CNNs in remote sensing remains the classification of hyperspectral images, where 2D-CNNs across the spatial dimension have also been tested [31], as well as 1D-CNNs across the spectral dimension [32], and even 3D-CNNs across spectral and spatial dimensions [33,34].”
Hyperspectral Image Classification—Traditional to Deep Models: A Survey for Future Prospects (2021, methodology): “A 3D-CNN model was introduced in [153] to jointly exploit spectral-spatial features from HSI and to validate its performance comparison is performed with spectral based DBN, SAE, and 2D-spatial CNN”
Spectral–Spatial Residual Network for Hyperspectral Image Classification: A 3-D Deep Learning Framework (2018, methodology): “The SSRN has a deeper structure than those of 3-D CNNs used in [21]–[24], and contains shortcut”
Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery (2018, background): “Recently, CNNs used for hyperspectral image classification have proven their effectiveness in extracting useful spectral–spatial features [28], [41].”
Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review (2019, background): “A similar approach was also described in [144,147] where spectral–spatial feature extraction and consequent classiﬁcation were done directly on hypercubes and without any pre-processing.”
A review of deep learning methods for semantic segmentation of remote sensing imagery (2021, methodology): “Yu et al. (2017) used 1 × 1 convolutional kernel to extract features from different bands and employed an average pooling layer and larger dropout rates in the CNN. Li et al. (2017) handles the hyperspectral cube with 3D CNNs, which processes the multiple bands with 3D convolutions.”
Deep Few-Shot Learning for Hyperspectral Image Classification (2019, background): “[43] have also demonstrated that small 3×3×3 kernels are an effective choice for 3-D CNN in HSI classification.”
3D Convolutional Neural Networks for Crop Classification with Multi-Temporal Remote Sensing Images (2018, background): “Recently, some studies have utilized 3D CNN for learning spatio-temporal features from videos [29,30], learning 3D structures from LiDAR point clouds [31], or learning spatio-spectral presentations from hyperspectral images [32].”
Hyperspectral Image Spatial Super-Resolution via 3D Full Convolutional Neural Network (2017, methodology): “The 3D convolution has been demonstrated to be very effective to explore volumetric data [54–57] and successfully applied to HSIs to explore both spatial context and spectral discrimination for classification [58].”
Deep Learning for Land Use and Land Cover Classification Based on Hyperspectral and Multispectral Earth Observation Data: A Review (2020, methodology): “In the case of remote sensing, 3D CNNs are used to extract spectral-spatial [114,115] and spatial-temporal [113] features.”
Deep learning in remote sensing: a review (2017, methodology): “The authors in [19] introduce a supervised, 2 , -regularized 3-D CNN-based model (see Figure 5), while the authors of [32] follow a similar idea for spatial-spectral classification.”
Deep learning for remote sensing image classification: A survey (2018, methodology): “Most recently, following the success of DL in natural image processing, DL methods have been introduced into HSI classification, achieving impressive results (Chen, Jiang, Li, Jia, & Ghamisi, 2016; Chen, Lin, Zhao, Wang, & Gu, 2017; Chen, Zhao, & Jia, 2015; Li, Zhang, & Shen, 2017).”
A review of deep learning used in the hyperspectral image analysis for agriculture (2021, background): “Some researchers regard the SAEs as a part of the framework to extract effective features (Chen and Li 2017).”
A Fast Dense Spectral-Spatial Convolution Network Framework for Hyperspectral Images Classification (2018, methodology): “(2) It has a deeper structure than the SSFC framework, and the methods in [11,12] which are CNN-based deep learning methods.”