Classification of Hyperspectral and LiDAR Data Using Coupled CNNs

By Renlong Hang, Zhu Li, Pedram Ghamisi, Danfeng Hong, Guiyu Xia, Qingshan Liu. IEEE Transactions on Geoscience and Remote Sensing, 2020

367 citations recorded by the source corpus.

In this article, we propose an efficient and effective framework to fuse hyperspectral and light detection and ranging (LiDAR) data using two coupled convolutional neural networks (CNNs). One CNN is designed to learn spectral–spatial features from hyperspectral data, and the other one is used to capture the elevation information from LiDAR data. Both of them consist of three convolutional layers, and the last two convolutional layers are coupled together via a parameter-sharing strategy. In the fusion phase, feature-level and decision-level fusion methods are simultaneously used to integrate these heterogeneous features sufficiently. For the feature-level fusion, three different fusion strategies are evaluated, including the concatenation strategy, the maximization strategy, and the summation strategy. For the decision-level fusion, a weighted summation strategy is adopted, where the weights are determined by the classification accuracy of each output. The proposed model is evaluated on an urban data set acquired over Houston, USA, and a rural one captured over Trento, Italy. On the Houston data, our model can achieve a new record overall accuracy (OA) of 96.03%. On the Trento data, it achieves an OA of 99.12%. These results sufficiently certify the effectiveness of our proposed model.

View Paper: https://www.semanticscholar.org/paper/787c00ffa08e8be7c45ca7d4089f3328e5549a17

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Extended Vision Transformer (ExViT) for Land Use and Land Cover Classification: A Multimodal Deep Learning Framework (2023, unknown): “…random forest (Stack-RF) [55]; 2) joint feature learning algorithms: CoSpace 7 [15], LeMA 8 [56]; 3) multi-modal CNN-based models: Co-CNN 9 [57], CCR-Net 10 [16]; and 4) transformer-based networks: Co-ViT, 11 Co-SF, 12 and DHViT [18], most of which have made their code publically available.”
Multimodal Fusion Transformer for Remote Sensing Image Classification (2022, methodology): “The random forest has been used due to its discriminative power when classifying unlabeled HSIs [28].”
Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review (2022, background): “, 2022), a feature-decision combined fusion network (Hang et al., 2020), and a”