Stacked Deconvolutional Network for Semantic Segmentation

By Jun Fu, J. Liu, Yuhang Wang, Hanqing Lu. 2017

223 citations recorded by the source corpus.

Recent progress in semantic segmentation has been driven by improving the spatial resolution under Fully Convolutional Networks (FCNs). To address this problem, we propose a Stacked Deconvolutional Network (SDN) for semantic segmentation. In SDN, multiple shallow deconvolutional networks, which are called as SDN units, are stacked one by one to integrate contextual information and bring the fine recovery of localization information. Meanwhile, inter-unit and intra-unit connections are designed to assist network training and enhance feature fusion since the connections improve the flow of information and gradient propagation throughout the network. Besides, hierarchical supervision is applied during the upsampling process of each SDN unit, which enhances the discrimination of feature representations and benefits the network optimization. We carry out comprehensive experiments and achieve the new state-ofthe- art results on four datasets, including PASCAL VOC 2012, CamVid, GATECH, COCO Stuff. In particular, our best model without CRF post-processing achieves an intersection-over-union score of 86.6% in the test set.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation (2019, methodology): “In addition to adopting those networks as backbones for semantic segmentation, one could employ the encoder-decoder structures [63, 2, 55, 44, 60, 58, 33, 78, 18, 11, 86, 82] which efficiently captures the long-range context information while keeping the detailed object boundaries.”
Searching for Efficient Multi-Scale Architectures for Dense Image Prediction (2018, methodology): “…objects are captured by the downsampled image [26, 66, 23, 50, 13, 11], (2) designing models that contain encoder-decoder structures [2, 72, 49, 28, 64, 93, 96], or (3) designing models that employ a multi-scale context module, e.g ., DenseCRF module [42, 4, 10, 98, 50, 76], global context…”
Deep High-Resolution Representation Learning for Visual Recognition (2019, background): “…skip connections [189] or exchanging information between them [49]; studying the details of the up-sample process [147]; combining multi-scale pyramid representations [22], [154]; stacking multiple DeconvNets/U-Nets/Hourglass [44], [149] with dense connections [135]. conv. unit . upsample Fig.”
ExFuse: Enhancing Feature Fusion for Semantic Segmentation (2018, neutral): “better than DeepLabv3 at the same setting. Method mIOU Tusimple [9] 83.1 Large Kernel Matters [8] 83.6 Multipath ReneNet [11] 84.2 ResNet 38 MS COCO [39] 84.9 PSPNet [10] 85.4 DeepLabv3 [5] 85.7 SDN [40] 86.6 DeepLabv3+ (Xception) [37] 87.8 ExFuse ResNet101 (ours) 86.2 ExFuse ResNeXt131 (ours) 87.9 ”
SPGNet: Semantic Prediction Guidance for Scene Parsing (2019, background): “Interestingly, we observe that the multi-stage encoder-decoder structure is seldom explored in the context of semantic segmentation, except [19, 51].”
DeeperLab: Single-Shot Image Parser (2019, background): “Typically, the encoder-decoder networks [48, 45, 46, 49, 50, 51, 52, 53, 54, 7, 55, 56] capture the context information in the encoder path and recover the object boundary in the decoder path.”
Affinity Derivation and Graph Merge for Instance Segmentation (2018, background): “have proposed Deeplabv3+ [8] by introducing an encoderdecoder structure [41, 36, 16, 27] to [7] which achieves promising performance.”
Image Segmentation Using Deep Learning: A Survey (2020, background): “Several other works adopt transposed convolutions, or encoder-decoders for image segmentation, such as Stacked Deconvolutional Network (SDN) [43], Linknet [44], W-”
Pyramid Stereo Matching Network (2018, methodology): “Moreover, stacked multiple encoder-decoder networks such as [5] and [20] were introduced to improve feature fusion.”
CycleSegNet: Object Co-Segmentation With Cycle Refinement and Region Correspondence (2021, background): “Although deep neural networks have shown remarkable success in many other segmentation tasks, such as semantic segmentation [7], [10], [16], [23], [26], [47], [48], interactive segmentation [36], and instance segmentation [12], [43], [49], their models can not be directly applied to the co-segmentat”
A Survey of Human-in-the-loop for Machine Learning (2021, background): “This field has recently become explosive popularity because it plays a crucial role in a wide range of computer vision applications [132, 133, 134, 135].”
Adaptive Context Network for Scene Parsing (2019, background): “Speciﬁcally, some meth-ods [22, 39, 34, 9] adopt “U-net” architectures, which ex-a r X i v : 1911 .”
Crowd Counting and Density Estimation by Trellis Encoder-Decoder Networks (2019, methodology): “Previous attempts have been made to provide multisupervision, where losses are computed between intermediate feature maps and the ground truths [48, 16, 8].”
Deep semantic segmentation of natural and medical images: a review (2019, background): “to the ﬁrst image segmentation network i.e. FCN. Several modiﬁed versions (e.g. deeper/shallower, adding extra attention blocks) of encoder-decoder networks have been applied to semantic segmentation [5,32,82,107,113, 155,170]. Recently in 2018, DeepLabV3+ [23] has outperformed many state-of-the-art”
Expectation-Maximization Attention Networks for Semantic Segmentation (2019, unknown): “Wide ResNet [32] WideResNet-38 84.9 PSPNet [37] ResNet-101 85.4 DeeplabV3 [4] ResNet-101 85.7 PSANet [38] ResNet-101 85.7 EncNet [35] ResNet-101 85.9 DFN [34] ResNet-101 86.2 Exfuse [36] ResNet-101 86.2 IDW-CNN [30] ResNet-101 86.3 SDN [12] DenseNet-161 86.6 DIS [23] ResNet-101 86.8”
FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation (2019, methodology): “Alter-natively, EncoderDecoders [24, 28, 18, 1, 26, 12, 33, 37] propose to utilize an encoder to extract multi-level feature maps, which are then combined into the ﬁnal prediction by a decoder.”
Squeeze-and-Attention Networks for Semantic Segmentation (2019, methodology): “tation from two independent dimensions — pixel-wise prediction and pixel grouping. We design a SA module to account for the Model Backbone mIoU MACs Params Dilated FCN ResNet101 78.7 162.7G 42.6M SDN [14] DenseNet 84.2 – 238.5M APCNet [16] ResNet101 83.5 – – Deeplab V3+y [8] ResNet101 81.5 235.6G 59”