Feedforward semantic segmentation with zoom-out features

By Mohammadreza Mostajabi, Payman Yadollahpour, Gregory Shakhnarovich. 2014

484 citations recorded by the source corpus.

We introduce a purely feed-forward architecture for semantic segmentation. We map small image elements (superpixels) to rich feature representations extracted from a sequence of nested regions of increasing extent. These regions are obtained by “zooming out” from the superpixel all the way to scene-level resolution. This approach exploits statistical structure in the image and in the label space without setting up explicit structured prediction mechanisms, and thus avoids complex and expensive inference. Instead superpixels are classified by a feedforward multilayer network. Our architecture achieves 69.6% average accuracy on the PASCAL VOC 2012 test set.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Fully convolutional networks for semantic segmentation (2014, background): “partial derivatives. In the context of deep networks, Sermanet et al. [28] fuse intermediate layers but discard resolution in doing so. In contemporary work Hariharan et al. [29] and Mostajabi et al. [30] also fuse multiple layers but do not learn end-to-end and rely on ﬁxed bottom-up grouping. FCN ”
The Cityscapes Dataset for Semantic Urban Scene Understanding (2016, background): “Other work takes advantage of deep learning for explicitly integrating global scene context in the prediction 5 of pixel-wise semantic labels, in particular through CNNs [4,39,44,67] or Recurrent Neural Networks (RNNs) [8,52].”
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (2016, methodology): “The second is to employ a super-pixel representation, essentially delegating the localization task to a lowlevel segmentation method [50].”
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs (2014, methodology): “This route is followed by the very successful recent method of Mostajabi et al. (2014).”
Pyramid Scene Parsing Network (2016, neutral): “2 Zoom-out [28] 85.”
Conditional Random Fields as Recurrent Neural Networks (2015, methodology): “The first strategy is based on utilizing separated mechanisms for both feature extraction, and image segmentation exploiting the edges of the image [2, 20].”
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (2018, background): “There are several model variants proposed to exploit the contextual information for segmentation [28, 65, 36, 39, 22, 79, 51, 14], including those that employ multi-scale inputs (i.”
Rethinking Atrous Convolution for Semantic Image Segmentation (2017, background): “2 for illustration) that exploit context information for semantic segmentation [26, 14, 54, 8, 85, 48, 57, 65, 78].”
Learning Representations for Automatic Colorization (2016, methodology): “We process a grayscale image through a deep convolutional architecture (VGG) [36] and take spatially localized multilayer slices (hypercolumns) [14, 25, 27], as per-pixel descriptors.”
Colorization as a Proxy Task for Visual Understanding (2017, background): “The networks use hypercolumns [23, 27, 9] with sparse training [21].”
ParseNet: Looking Wider to See Better (2015, background): “[28, 21] show that simply adding such context features helps in the detection and segmentation tasks.”
Learning Deconvolution Network for Semantic Segmentation (2015, background): “Convolutional neural networks (CNNs) are widely used in various visual recognition problems such as image classification [17, 24, 25], object detection [8, 10], semantic segmentation [7, 20], visual tracking [13], and action recognition [15, 23].”
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation (2015, methodology): “For CNN-based methods, background information can be effectively captured by combining features from a multi-scale image network input, and has shown good performance in some recent segmentation methods [13, 33].”
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation (2015, background): “The common idea is to use features extracted at multiple scales to provide both local and global context [45]”
Semantic Image Segmentation via Deep Parsing Network (2015, methodology): “With deep models, these works [23, 24, 25, 22, 3, 28, 39, 30, 19] demonstrated encouraging results using simple definition of the pairwise function or even ignore it.”
Attention to Scale: Scale-Aware Semantic Image Segmentation (2015, methodology): “[41] classified a superpixel with features extracted at zoom-out spatial levels from a small proximal neighborhood to the whole image region.”
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation (2015, methodology): “This model is also used by all competitors [19, 11, 5, 4, 20] compared in this paper.”
Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network (2017, methodology): “Among the ﬁrst, Zoom-out [26] proposes a hand-crafted hierarchical context features, while ParseNet [23] adds a global pooling branch to extract context information.”
Recent advances in convolutional neural networks (2015, methodology): “[191] apply the local and proximal features from a ConvNet and apply the Alex-net [6] to obtain the distant and global features, and their concatenation gives rise to the zoomout features.”
Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation (2019, background): “It has been known that context information is crucial for pixel labeling tasks [26, 70, 37, 39, 16, 54, 14, 10].”