Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning

By Ronald Kemker, Carl Salvaggio, Christopher Kanan. Isprs Journal of Photogrammetry and Remote Sensing, 2017

527 citations recorded by the source corpus.

Abstract Deep convolutional neural networks (DCNNs) have been used to achieve state-of-the-art performance on many computer vision tasks (e.g., object recognition, object detection, semantic segmentation) thanks to a large repository of annotated image data. Large labeled datasets for other sensor modalities, e.g., multispectral imagery (MSI), are not available due to the large cost and manpower required. In this paper, we adapt state-of-the-art DCNN frameworks in computer vision for semantic segmentation for MSI imagery. To overcome label scarcity for MSI data, we substitute real MSI for generated synthetic MSI in order to initialize a DCNN framework. We evaluate our network initialization scheme on the new RIT-18 dataset that we present in this paper. This dataset contains very-high resolution MSI collected by an unmanned aircraft system. The models initialized with synthetic imagery were less prone to over-fitting and provide a state-of-the-art baseline for future work.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/0123f7606e0128af5eb57125f2fcfb14a967259f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery (2020, background): “mote sensing imagery is very helpful for urban management, planing and monitoring [39, 40, 25, 26].”
A review of deep learning methods for semantic segmentation of remote sensing imagery (2021, methodology): “Alternatively, Kemker et al. (2018) presented a stacked convolutional auto-encoder model to extract features for HSI classification.”
Research Progress on Few-Shot Learning for Remote Sensing Image Interpretation (2021, methodology): “Kemker et al. [35] investigated using the simulation techniques to generate synthetic imagery as the source domain to provide the pretrained models for multispectral remote sensing images.”
Survey on semantic segmentation using deep learning techniques (2019, background): “…in computer vision & artiﬁcial intelligence − autonomous driving [2, 3], robot navigation [4], industrial inspection [5]; remote sensing [6]; In cognitive and computational sciences − saliency object detection [7, 8]; In Agriculture sciences [9]; Fashion − categorizing clothing items…”
Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks (2023, methodology): “The CNNs-based architecture was also used in [5] for semantic segmentation on multispectral RS images rather than high-resolution RGB images.”
Deep learning for change detection in remote sensing: a review (2022, background): “…Zhang et al. 2020; Cheng et al. 2020), change detection (Zhu et al. 2018), land use and cover classification (Hong et al. 2021, 2020b, 2020a; Li et al. 2022; Zhou et al. 2022), semantic segmentation (Kemker, Salvaggio, and Kanan 2018), and object-based image analysis (Liu, Yang, and Lunga 2021).”
Diverse Region-Based CNN for Hyperspectral Image Classification (2018, methodology): “Deep learning methods [10], [23]–[29], which act more dynamically to provide automation features, have been extensively employed for remote sensing image feature extraction and classiﬁcation.”
Semantic segmentation of slums in satellite images using transfer learning on fully convolutional neural networks (2019, unknown): “Expanding the three channel input limitations of traditional deep learning algorithms (Kemker et al., 2018; Marmanis et al., 2018) use specific architectures to use elevation information and multispectral imagery to boost performance in semantic segmentation frameworks.”
Deep Learning for Land Use and Land Cover Classification Based on Hyperspectral and Multispectral Earth Observation Data: A Review (2020, methodology): “Conditional random fields (CRFs) is the main technique used for this end [178], and it has been successfully practised by many studies jointly with deep learning models targeting semantically segmented maps [75,179,180].”
UNetFormer: A UNet-like transformer for efficient semantic segmentation of remote sensing urban scene imagery (2021, unknown): “Since then, CNN-based methods have dominated the semantic segmentation task in the remote sensing field (Kemker et al., 2018; Kotaridis and Lazaridou, 2021; Ma et al., 2019; Tong et al., 2020; Zhao and Du, 2016; Zhu et al., 2017).”