Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks

By Nicolas Audebert, Bertrand Le Saux, Sébastien Lefèvre. Isprs Journal of Photogrammetry and Remote Sensing, 2017

621 citations recorded by the source corpus.

In this work, we investigate various methods to deal with semantic labeling of very high resolution multi-modal remote sensing data. Especially, we study how deep fully convolutional networks can be adapted to deal with multi-modal and multi-scale remote sensing data for semantic labeling. Our contributions are threefold: a) we present an efficient multi-scale approach to leverage both a large spatial context and the high resolution data, b) we investigate early and late fusion of Lidar and multispectral data, c) we validate our methods on two public datasets with state-of-the-art results. Our results indicate that late fusion make it possible to recover errors steaming from ambiguous data, while early fusion allows for better joint-feature learning but at the cost of higher sensitivity to missing data.

Subfields: Ecology, Environmental Engineering, Media Technology

View Paper: https://www.semanticscholar.org/paper/de6b93fdcf5007482ab080d4eed61e97a8b9dda8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Ecology

In Environmental Engineering

In Media Technology

Citations in the corpus

TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-captured Scenarios (2021, background): “Object detection technology on drone-captured scenarios has been widely used in many practical applications, such as plant protection [18, 41], wildlife protection [23, 22] and urban surveillance [1, 15].”
A Novel Transformer Based Semantic Segmentation Scheme for Fine-Resolution Remote Sensing Images (2021, methodology): “Following previous pieces of literature [5, 12, 14], in the Vaihingen dataset, we use the benchmark organizer defined 16 images for training and 17 for testing, while the setting in the Potsdam dataset is 24 tiles for training and 14 tiles for testing.”
A Multilevel Multimodal Fusion Transformer for Remote Sensing Semantic Segmentation (2024, methodology): “Inspired by FuseNet, vFuseNet [15] was established to leverage multiscale fusion strategies.”
Fully Convolutional Siamese Networks for Change Detection (2018, background): “Despite achieving state-of-the-art results in other Earth observation problems [17], these techniques have not yet been applied to CD to the best of our knowledge.”
A review of deep learning methods for semantic segmentation of remote sensing imagery (2021, methodology): “In addition to the common practice of using image tiles as training examples, researchers also used a sliding window technique to create training images of smaller size but with redundancy (Audebert et al., 2018; Xu et al., 2018).”
More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing Imagery Classification (2020, background): “The same researchers further extended their model with two kinds of fusion strategies: early fusion and late fusion [38].”
UNetFormer: A UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery (2021, positive): “, 2021b), (iii) CNN-based networks for semantic segmentation of remote sensing images: DST_5 (Sherrah, 2016), V-FuseNet (Audebert et al., 2018), CASIA2 (Liu et al.”
Unmanned Aerial Vehicle for Remote Sensing Applications - A Review (2019, background): “For multi-modal data integration, deep convolution neural networks also show their capacity in both feature level fusion [112] and decision level fusion [113] for increased classiﬁcation accuracy.”