PixelNet: Towards a General Pixel-level Architecture

By Aayush Bansal, Xinlei Chen, Bryan C. Russell, Abhinav Kumar Gupta, Deva Ramanan. 2016

63 citations recorded by the source corpus.

We explore architectures for general pixel-level prediction problems, from low-level edge detection to mid-level surface normal estimation to high-level semantic segmentation. Convolutional predictors, such as the fully-convolutional network (FCN), have achieved remarkable success by exploiting the spatial redundancy of neighboring pixels through convolutional processing. Though computationally efficient, we point out that such approaches are not statistically efficient during learning precisely because spatial redundancy limits the information learned from neighboring pixels. We demonstrate that (1) stratified sampling allows us to add diversity during batch updates and (2) sampled multi-scale features allow us to explore more nonlinear predictors (multiple fully-connected layers followed by ReLU) that improve overall accuracy. Finally, our objective is to show how a architecture can get performance better than (or comparable to) the architectures designed for a particular task. Interestingly, our single architecture produces state-of-the-art results for semantic segmentation on PASCAL-Context, surface normal estimation on NYUDv2 dataset, and edge detection on BSDS without contextual post-processing.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

Citations in the corpus

An Implementation of Faster RCNN with Study for Region Sampling (2017, methodology): “A proper bias in sampling is known to help at least converge networks more quickly [1] and is actually also used in Faster RCNN: a ﬁxed positive/negative ratio to avoid always learning on negative patches.”
Singing Voice Separation with Deep U-Net Convolutional Networks (2017, background): “Over the last few years, considerable improvements have occurred in the family of machine learning algorithms known as image-to-image translation [11] — pixel-level classification [2], automatic colorization [33], image segmentation [1] — largely driven by advances in the design of novel neural netw”
Enhancement of SSD by concatenating feature maps for object detection (2017, background): “This paper is inspired by the fact that we get improved abstract representation of the original image from features in different scales [1, 5].”
Deep Level Sets for Salient Object Detection (2017, background): “Also as discussed in [5], for dense pixel-labeling tasks, the training of networks is always based on the assumption that pixels are independent.”
Episodic CAMN: Contextual Attention-Based Memory Networks with Iterative Feedback for Scene Labeling (2017, unknown): “0 PixelNet [4] - 51.”
Finding Tiny Faces (2016, background): “[8, 14, 1] show that deep multi-scale descriptors (known as “hypercolumns”) are useful for semantic segmentation.”
Predicting Ground-Level Scene Layout from Aerial Imagery (2016, methodology): “Aerial Image Feature Extraction For A ( I a ; Θ A ) , we use the VGG16 [27] base architecture and convert it to a pixel-level labeling method using the PixelNet approach [3].”
Fast Face-Swap Using Convolutional Neural Networks (2016, methodology): “While fast and relatively accurate methods for segmentation exist, including some based on neural networks [1, 19, 22], we assume for simplicity that a segmentation mask is given and focus on the remaining problems.”
Loss Max-Pooling for Semantic Image Segmentation (2017, result): “In our experiments on the Cityscapes and Pascal VOC 2012 segmentation datasets we find consistently improved results, demonstrating the efficacy of our approach.”
Exploiting Saliency for Object Segmentation from Image Level Labels (2017, background): “Multiple convnet architectures have been proposed, including recurrent networks [34], encoder-decoders [29, 1], up-sampling layers [27], using skip layers [2], or dilated convolutions [7, 49], to name a few.”