Combining the Best of Convolutional Layers and Recurrent Layers: A Hybrid Network for Semantic Segmentation

By Zhicheng Yan, H. Zhang, Yangqing Jia, Thomas Breuel, Yizhou Yu. 2016

42 citations recorded by the source corpus.

State-of-the-art results of semantic segmentation are established by Fully Convolutional neural Networks (FCNs). FCNs rely on cascaded convolutional and pooling layers to gradually enlarge the receptive fields of neurons, resulting in an indirect way of modeling the distant contextual dependence. In this work, we advocate the use of spatially recurrent layers (i.e. ReNet layers) which directly capture global contexts and lead to improved feature representations. We demonstrate the effectiveness of ReNet layers by building a Naive deep ReNet (N-ReNet), which achieves competitive performance on Stanford Background dataset. Furthermore, we integrate ReNet layers with FCNs, and develop a novel Hybrid deep ReNet (H-ReNet). It enjoys a few remarkable properties, including full-image receptive fields, end-to-end training, and efficient network execution. On the PASCAL VOC 2012 benchmark, the H-ReNet improves the results of state-of-the-art approaches Piecewise, CRFasRNN and DeepParsing by 3.6%, 2.3% and 0.2%, respectively, and achieves the highest IoUs for 13 out of the 20 object classes.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Rethinking Atrous Convolution for Semantic Image Segmentation (2017, methodology): “There are other methods based on LSTM [31] to aggregate global context [46, 6, 79].”
ReSeg: A Recurrent Neural Network-Based Model for Semantic Segmentation (2015, methodology): “1Subsequent but independent work [47] investigated the combination of ReSeg with Fully Convolutional Network (FCN) and CRFs, reporting state of the art results on Pascal VOC.”
Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks (2016, background): “ResNet [11] based architectures [12], recurrent neural networks [13]).”
Episodic CAMN: Contextual Attention-Based Memory Networks with Iterative Feedback for Scene Labeling (2017, background): “However, it would be ineffective to enlarge the receptive fields of the neurons to explicitly and directly model long-range context as it may degrade the local discriminative features [66].”
Deep Learning Advances in Computer Vision with 3D Data (2017, methodology): “This indication was later conﬁrmed by other works as well (e.g., Byeon et al. [2015] and Yan et al. [2016] successfully used spatial RNNs for semantic segmentation, while Bell et al. [2015] exploited them for object detection).”