Convolutional Neural Networks for Multimodal Remote Sensing Data Classification

By Xin Wu, Danfeng Hong, Jocelyn Chanussot. IEEE Transactions on Geoscience and Remote Sensing, 2022

317 citations recorded by the source corpus.

In recent years, enormous research has been made to improve the classification performance of single-modal remote sensing (RS) data. However, with the ever-growing availability of RS data acquired from satellite or airborne platforms, simultaneous processing and analysis of multimodal RS data pose a new challenge to researchers in the RS community. To this end, we propose a deep-learning-based new framework for multimodal RS data classification, where convolutional neural networks (CNNs) are taken as a backbone with an advanced cross-channel reconstruction module, called CCR-Net. As the name suggests, CCR-Net learns more compact fusion representations of different RS data sources by the means of the reconstruction strategy across modalities that can mutually exchange information in a more effective way. Extensive experiments conducted on two multimodal RS datasets, including hyperspectral (HS) and light detection and ranging (LiDAR) data, i.e., the Houston2013 dataset, and HS and synthetic aperture radar (SAR) data, i.e., the Berlin dataset, demonstrate the effectiveness and superiority of the proposed CCR-Net in comparison with several state-of-the-art multimodal RS data classification methods. The codes will be openly and freely available at https://github.com/danfenghong/IEEE_TGRS_CCR-Net for the sake of reproducibility.

Subfields: Atmospheric Science, Ecology, Media Technology

View Paper: https://www.semanticscholar.org/paper/10a7c1617189f013ea4549d4abe05f20f4e6073f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Atmospheric Science

1. Danfeng Hong
2. Jing Yao
3. Jocelyn Chanussot
4. Lianru Gao
5. Bing Zhang
6. Jon Atli Benediktsson
7. Shutao Li
8. Jun Yu Li

In Ecology

1. Danfeng Hong
2. Jing Yao
3. Bing Zhang
4. Man On Pun
5. Xiaokang Zhang
6. Xianping Ma
7. Ming Liu
8. Jocelyn Chanussot

In Media Technology

1. Danfeng Hong
2. Jocelyn Chanussot
3. Jing Yao
4. Bing Zhang
5. Lianru Gao
6. Man On Pun
7. Xiaokang Zhang
8. Xianping Ma

Citations in the corpus

• Extended Vision Transformer (ExViT) for Land Use and Land Cover Classification: A Multimodal Deep Learning Framework (2023, methodology): “Besides, the other CNN-based method CCR-Net surprisingly achieves the best performance in seven classes owing to its elaborate design of an effective cross-channel reconstruction module.”
• Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review (2022, background): “com/Cimy-wang/AM3Net Multimodal Data Fusion CCR-Net (Wu et al., 2022) TensorFlow https://github.”
• A Multilevel Multimodal Fusion Transformer for Remote Sensing Semantic Segmentation (2024, methodology): “Recently, deep learning (DL)-based methods have been successfully developed for remote sensing semantic segmentation by exploiting convolutional neural networks (CNNs) [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25].”