Deep hierarchical parsing for semantic segmentation

By Abhishek Sharma, Oncel Tuzel, David W. Jacobs. 2015

117 citations recorded by the source corpus.

This paper proposes a learning-based approach to scene parsing inspired by the deep Recursive Context Propagation Network (RCPN). RCPN is a deep feed-forward neural network that utilizes the contextual information from the entire image, through bottom-up followed by top-down context propagation via random binary parse trees. This improves the feature representation of every super-pixel in the image for better classification into semantic categories. We analyze RCPN and propose two novel contributions to further improve the model. We first analyze the learning of RCPN parameters and discover the presence of bypass error paths in the computation graph of RCPN that can hinder contextual propagation. We propose to tackle this problem by including the classification loss of the internal nodes of the random parse trees in the original RCPN loss function. Secondly, we use an MRF on the parse tree nodes to model the hierarchical dependency present in the output. Both modifications provide performance boosts over the original RCPN and the new system achieves state-of-the-art performance on Stanford Background, SIFT-Flow and Daimler urban datasets.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

The Cityscapes Dataset for Semantic Urban Scene Understanding (2016, background): “Other work takes advantage of deep learning for explicitly integrating global scene context in the prediction 5 of pixel-wise semantic labels, in particular through CNNs [4,39,44,67] or Recurrent Neural Networks (RNNs) [8,52].”
Multi-scale context for scene labeling via flexible segmentation graph (2016, methodology): “Additionally, we plan to embed deep learning algorithms, such as [14,15], into our multi-scale framework to improve the performance, while retaining high efficiency.”
The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes (2017, background): “Others, shape the architecture to explicitly integrate global context in the pixelwise classifier, or ensemble features at multiple scales, either through CNNs [2, 35, 38, 40, 51, 57, 61] or”
Episodic CAMN: Contextual Attention-Based Memory Networks with Iterative Feedback for Scene Labeling (2017, background): “[49,50] exploit the recursive neural network architecture to propagate context.”
Recent progress in semantic image segmentation (2018, background): “Researchers also exploit segmentation by using super-pixels (Mostajabi et al. 2015; Sharma et al. 2015).”
SegNBDT: Visual Decision Rules for Segmentation (2020, background): “The corresponding deep learning reincarnations for segmentation integrate convolutional filters [11] or whole neural networks into each decision rule [22, 7], and other approaches define hierarchies spatially, hierarchically merging pixels or superpixels [26, 13, 30].”
Effective Use of Synthetic Data for Urban Scene Semantic Segmentation (2018, background): “Semantic segmentation, that is, understanding an image at pixel-level, has been widely studied by the computer vision community [43, 46, 15, 26, 11, 33, 42, 25, 28, 4, 53, 52].”
Gaussian Conditional Random Field Network for Semantic Segmentation (2016, background): “Building on top of these CNN features, [46, 47] introduced a recursive context propagation network that enriched the CNN features by adding image level contextual information.”
PointGrid: A Deep Network for 3D Shape Understanding (2018, methodology): “Deep learning has become a universal tool for many visual recognition tasks ranging from classification to segmentation, especially ConvNets for 2D images [28, 48, 50, 16, 10, 34, 37, 45, 18] thanks to its weight sharing and other kernel optimizations of 2D convolutions.”
Region-Based Semantic Segmentation with End-to-End Training (2016, methodology): “[32,33]) as feature representations [2,4,6,7,9,11,12].”
STFCN: Spatio-Temporal FCN for Semantic Video Segmentation (2016, methodology): “The support vector machine (SVM) [43] or a Markov random ﬁeld (MRF) [38,46] are further methods used for segmentation but not as popular as the conditional random ﬁeld (CRF) that is in widespread use in recent work[5,29,35].”