Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review

By Jiaxin Li, Danfeng Hong, Lianru Gao, Jing Yao, Ke-xin Zheng, Bing Zhang, Jocelyn Chanussot. International Journal of Applied Earth Observation and Geoinformation, 2022

692 citations recorded by the source corpus.

With the extremely rapid advances in remote sensing (RS) technology, a great quantity of Earth observation (EO) data featuring considerable and complicated heterogeneity is readily available nowadays, which renders researchers an opportunity to tackle current geoscience applications in a fresh way. With the joint utilization of EO data, much research on multimodal RS data fusion has made tremendous progress in recent years, yet these developed traditional algorithms inevitably meet the performance bottleneck due to the lack of the ability to comprehensively analyse and interpret these strongly heterogeneous data. Hence, this non-negligible limitation further arouses an intense demand for an alternative tool with powerful processing competence. Deep learning (DL), as a cutting-edge technology, has witnessed remarkable breakthroughs in numerous computer vision tasks owing to its impressive ability in data representation and reconstruction. Naturally, it has been successfully applied to the field of multimodal RS data fusion, yielding great improvement compared with traditional methods. This survey aims to present a systematic overview in DL-based multimodal RS data fusion. More specifically, some essential knowledge about this topic is first given. Subsequently, a literature survey is conducted to analyse the trends of this field. Some prevalent sub-fields in the multimodal RS data fusion are then reviewed in terms of the to-be-fused data modalities, i.e., spatiospectral, spatiotemporal, light detection and ranging-optical, synthetic aperture radar-optical, and RS-Geospatial Big Data fusion. Furthermore, We collect and summarize some valuable resources for the sake of the development in multimodal RS data fusion. Finally, the remaining challenges and potential future directions are highlighted.

Subfields: Artificial Intelligence, Environmental Engineering, Media Technology

View Paper: https://www.semanticscholar.org/paper/2dabebdca6b3211c3950be2c4a8c9b39b8637ece

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Lina Zhuang
2. Antonio J. Plaza
3. Man On Pun
4. Degang Wang
5. Xiaokang Zhang
6. Xu Sun
7. Xianping Ma
8. Min Huang

In Environmental Engineering

1. Antonio J. Plaza

In Media Technology

1. Lina Zhuang
2. Lianru Gao
3. Antonio J. Plaza
4. Man On Pun
5. Degang Wang
6. Xiaokang Zhang
7. Xu Sun
8. Xianping Ma

Citations in the corpus

• A Multilevel Multimodal Fusion Transformer for Remote Sensing Semantic Segmentation (2024, background): “Compared with natural images, high-resolution remote sensing images have more severe spectral heterogeneity and more complex spatial structures [3], [35].”
• BockNet: Blind-Block Reconstruction Network With a Guard Window for Hyperspectral Anomaly Detection (2023, background): “H YPERSPECTRAL remote-sensing images depict both the spectral and spatial characteristics of ground objects [1], [2] and this detailed information provides valuable data for identifying ground materials [3], [4].”