Learning Deconvolution Network for Semantic Segmentation

By Hyeonwoo Noh, Seunghoon Hong, Bohyung Han. 2015

4,362 citations recorded by the source corpus.

We propose a novel semantic segmentation algorithm by learning a deep deconvolution network. We learn the network on top of the convolutional layers adopted from VGG 16-layer net. The deconvolution network is composed of deconvolution and unpooling layers, which identify pixelwise class labels and predict segmentation masks. We apply the trained network to each proposal in an input image, and construct the final semantic segmentation map by combining the results from all proposals in a simple manner. The proposed algorithm mitigates the limitations of the existing methods based on fully convolutional networks by integrating deep deconvolution network and proposal-wise prediction, our segmentation method typically identifies detailed structures and handles objects in multiple scales naturally. Our network demonstrates outstanding performance in PASCAL VOC 2012 dataset, and we achieve the best accuracy (72.5%) among the methods trained without using Microsoft COCO dataset through ensemble with the fully convolutional network.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Fully convolutional networks for semantic segmentation (2014, background): “model is an end-to-end integration of the CRF for further improvement. ParseNet [41] normalizes features for fusion and captures context with global pooling. The “deconvolutional network” approach of [42] restores resolution by proposals, stacks of learned deconvolution, and unpooling. U-Net [43] co”
Unified Perceptual Parsing for Scene Understanding (2018, background): “Noh et al. [12] propose a framework which applies deconvolution [13] to up-sample low resolution feature maps.”
Semantic Understanding of Scenes Through the ADE20K Dataset (2016, methodology): “eural networks (CNN) for image classiﬁcation [17], there is growing interest for semantic pixel-wise labeling using CNNs with dense output, such as the fully CNN [20], deconvolutional neural networks [25], encoderdecoder SegNet [1], multi-task network cascades [9], and DilatedVGG [5, 36]. They are b”
Fully Convolutional Instance-Aware Semantic Segmentation (2016, background): “The FCNs are extended with global context [28], multi-scale feature fusion [4], and deconvolution [31].”
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation (2016, methodology): “In [14], a pre-trained VGG network architecture [19] was used in conjunction with its mirrored, de-convolutional equivalent to segment RGB images, by leveraging the descriptive power of the features extracted by the innermost layer.”
Learning to decompose for object detection and instance segmentation (2015, background): “The proposed network, depicted in figure 1, is an all convolutional network inspired by Springenberg et al. (2015); Lin et al. (2013); Noh et al. (2015).”
Beyond Skip Connections: Top-Down Modulation for Object Detection (2016, methodology): “Deconvolutional [36, 55] and encoder-decoder networks [3] are often used to generate such finer features by using the ‘unpool’ operation.”
Learning to Refine Object Segments (2016, methodology): “Deconv networks have recently been applied successfully to semantic segmentation [19].”
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation (2016, background): “In order to compensate for the resolution loss induced by pooling layers, FCNs introduce skip connections between their downsampling and upsampling paths.”
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (2016, background): “This direction has been extended by several follow-up papers [17], [40], [58], [59], [60], [61], [62], [63], [65], since the first version of our work was published [38].”
Perceptual Losses for Real-Time Style Transfer and Super-Resolution (2016, methodology): “The architecture of our transformation networks are inspired by [4] and [16], which use in-network downsampling to reduce the spatial extent of feature maps followed by in-network upsampling to produce the final output image.”
Multi-Scale Context Aggregation by Dilated Convolutions (2015, background): “One approach involves repeated up-convolutions that aim to recover lost resolution while carrying over the global perspective from downsampled layers (Noh et al., 2015; Fischer et al., 2015).”
Pyramid Scene Parsing Network (2016, background): “[30] proposed a coarse-to-fine structure with deconvolution network to learn the segmentation mask.”
Scene Parsing through ADE20K Dataset (2017, background): “With the success of convolutional neural networks (CNN) for image classification [16], there is growing interest for semantic image parsing using CNNs with dense output, such as the multiscale CNN [11], recurrent CNN [25], fully CNN [19], deconvolutional neural networks [24], encoder-decoder SegNet ”
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (2018, methodology): “Encoder-decoder: The encoder-decoder networks have been successfully applied to many computer vision tasks, including human pose estimation [53], object detection [45, 66, 19], and semantic segmentation [49, 54, 61, 3, 43, 59, 57, 33, 76, 20].”
Rethinking Atrous Convolution for Semantic Image Segmentation (2017, methodology): “Deconvolutional layers (or transposed convolution) [79, 51, 54, 3, 61, 58] have been employed to recover the spatial resolution.”
Path Aggregation Network for Instance Segmentation (2018, methodology): “[46] fused information from lower layers through skip-connections.”
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions (2021, background): “[46] and achieved impressive performance on the PASCAL VOC 2012 dataset [51].”
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation (2015, background): “, training deconvolution networks [35], training multiple coarse to fine learning networks [9], and exploring middle layer features for high-resolution prediction [18, 32].”