Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks

By Michele Volpi, Devis Tuia. IEEE Transactions on Geoscience and Remote Sensing, 2016

505 citations recorded by the source corpus.

Semantic labeling (or pixel-level land-cover classification) in ultrahigh-resolution imagery (<10 cm) requires statistical models able to learn high-level concepts from spatial data, with large appearance variations. Convolutional neural networks (CNNs) achieve this goal by learning discriminatively a hierarchy of representations of increasing abstraction. In this paper, we present a CNN-based system relying on a downsample-then-upsample architecture. Specifically, it first learns a rough spatial map of high-level representations by means of convolutions and then learns to upsample them back to the original resolution by deconvolutions. By doing so, the CNN learns to densely label every pixel at the original resolution of the image. This results in many advantages, including: 1) the state-of-the-art numerical accuracy; 2) the improved geometric accuracy of predictions; and 3) high efficiency at inference time. We test the proposed system on the Vaihingen and Potsdam subdecimeter resolution data sets, involving the semantic labeling of aerial images of 9- and 5-cm resolution, respectively. These data sets are composed by many large and fully annotated tiles, allowing an unbiased evaluation of models making use of spatial information. We do so by comparing two standard CNN architectures with the proposed one: standard patch classification, prediction of local label patches by employing only convolutions, and full patch labeling by employing deconvolutions. All the systems compare favorably or outperform a state-of-the-art baseline relying on superpixels and powerful appearance descriptors. The proposed full patch labeling CNN outperforms these models by a large margin, also showing a very appealing inference time.

Subfields: Computer Vision and Pattern Recognition, Media Technology, Ocean Engineering

View Paper: https://www.semanticscholar.org/paper/ef500cd3c401cd0632656c9fddccb1655c6659f9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

In Ocean Engineering

Citations in the corpus

RSVQA: Visual Question Answering for Remote Sensing Data (2020, background): “Volpi and Tuia [14] evaluated different strategies for segment-”
Unsupervised Deep Change Vector Analysis for Multiple-Change Detection in VHR Images (2019, background): “The number of features obtained from each layer is high (up to 512 for the model provided in [46]).”
Multitask learning for large-scale semantic change detection (2018, background): “FCNNs currently achieve state-ofthe-art results in semantic segmentation problems, including those in remote sensing (Volpi and Tuia, 2017; Maggiori et al., 2017; Chen et al., 2018a).”
Topological Map Extraction From Overhead Images (2018, background): “Today, most building footprint delineation from a single image is often approached via semantic segmentation as part of a broader multi-class task and many works exist, e.g., [40, 24, 30, 51, 20, 31].”
Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks (2017, unknown): “Lately, semantic labeling of aerial images has moved to FCN models [5, 4, 32].”
Deep learning in remote sensing: a review (2017, methodology): “◗ Deconvolution network in MatConvNet: The first example is released by the authors of [92] and corresponds to the architecture in Figure 11.”
Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series (2018, methodology): “CNNs have been widely applied to various remote sensing tasks including land cover classification of very high spatial resolution images [25,26], semantic segmentation [27], object detection [28], reconstruction of missing data [29], or pansharpening [30].”
Multilayer Convolution Neural Network for the Classification of Mango Leaves Infected by Anthracnose Disease (2019, background): “There are various adjustable parameters like the dropout rate, the learning rate used in complex processing for solving classification and pattern recognition problems [24], [25].”
Self-Attention for Raw Optical Satellite Time Series Classification (2019, background): “Following these developments, mono-temporal Earth observation approaches have adapted 2D convolutional neural networks from computer vision with great success [35, 36, 37, 38].”
Deep Learning for Classification of Hyperspectral Data: A Comparative Review (2019, unknown): “In particular, standard optical imagery (Red-Green-Blue –RGB– and Infra-Red –IR–) beneﬁted from using deep convolutional neural networks (CNN) for tasks such as classiﬁcation, object detection or semantic segmentation [1], [2], [3].”
Continual Learning With Structured Inheritance for Semantic Segmentation in Aerial Imagery (2021, methodology): “Following the prior works [57]–[59], we select ﬁve validation images from the training set.”
Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark (2017, background): “In the context of high-resolution image classification, the Vaihingen and Potsdam datasets [3] have gained increasing attention over the last year.”