Gated Convolutional Neural Network for Semantic Segmentation in High-Resolution Images

By Hongzhen Wang, Ying Wang, Qian Zhang, Shiming Xiang, Chunhong Pan. 2017

185 citations recorded by the source corpus.

Semantic segmentation is a fundamental task in remote sensing image processing. The large appearance variations of ground objects make this task quite challenging. Recently, deep convolutional neural networks (DCNNs) have shown outstanding performance in this task. A common strategy of these methods (e.g., SegNet) for performance improvement is to combine the feature maps learned at different DCNN layers. However, such a combination is usually implemented via feature map summation or concatenation, indicating that the features are considered indiscriminately. In fact, features at different positions contribute differently to the final performance. It is advantageous to automatically select adaptive features when merging different-layer feature maps. To achieve this goal, we propose a gated convolutional neural network to fulfill this task. Specifically, we explore the relationship between the information entropy of the feature maps and the label-error map, and then a gate mechanism is embedded to integrate the feature maps more effectively. The gate is implemented by the entropy maps, which are generated to assign adaptive weights to different feature maps as their relative importance. Generally, the entropy maps, i.e., the gates, guide the network to focus on the highly-uncertain pixels, where detailed information from lower layers is required to improve the separability of these pixels. The selected features are finally combined to feed into the classifier layer, which predicts the semantic label of each pixel. The proposed method achieves competitive segmentation accuracy on the public ISPRS 2D Semantic Labeling benchmark, which is challenging for segmentation by only using the RGB images.

Subfields: Computer Vision and Pattern Recognition, Media Technology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Free-Form Image Inpainting With Gated Convolution (2018, background): “Feature-wise gating has been explored widely in vision [12, 28, 39, 41], language [6], speech [27] and many other tasks.”
Building Extraction in Very High Resolution Remote Sensing Imagery Using Deep Learning and Guided Filters (2018, background): “At the same time, the overall accuracy (OA) can be obtained by normalizing the trace from the confusion matrix [52].”
A review of deep learning methods for semantic segmentation of remote sensing imagery (2021, methodology): “…cross-entropy as the loss function (Alshehhi et al., 2017; Cheng et al., 2017; Fu et al., 2017; Kussul et al., 2017; Li, Zhang, & Shen, 2017; Wang et al., 2017; Yu, Jia, & Xu, 2017; Chen et al., 2018; Chen et al., 2018; Chen et al., 2018; Kestur et al., 2018; Liu et al., 2018; Liu, Yu, Yu,…”
Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning (2017, methodology): “…have used DCNNs pre-trained on ImageNet to yield state-of-the-art results for the semantic segmentation of highresolution multispectral aerial imagery (Wang et al., 2017; Marmanis et al., 2016) because the most widely used benchmarks (Rottensteiner et al., 2012) only use a single non-RGB band.”