Proposal-Free Network for Instance-Level Object Segmentation

By Xiaodan Liang, Liang Lin, Yunchao Wei, Xiaohui Shen, Jianchao Yang, Shuicheng Yan. 2015

238 citations recorded by the source corpus.

Instance-level object segmentation is an important yet under-explored task. Most of state-of-the-art methods rely on region proposal methods to extract candidate segments and then utilize object classification to produce final results. Nonetheless, generating reliable region proposals itself is a quite challenging and unsolved task. In this work, we propose a Proposal-Free Network (PFN) to address the instance-level object segmentation problem, which outputs the numbers of instances of different categories and the pixel-level information on i) the coordinates of the instance bounding box each pixel belongs to, and ii) the confidences of different categories for each pixel, based on pixel-to-pixel deep convolutional neural network. All the outputs together, by using any off-the-shelf clustering method for simple post-processing, can naturally generate the ultimate instance-level object segmentation results. The whole PFN can be easily trained without the requirement of a proposal generation stage. Extensive evaluations on the challenging PASCAL VOC 2012 semantic segmentation benchmark demonstrate the effectiveness of the proposed PFN solution without relying on any proposal generation methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Recurrent Neural Networks for Semantic Instance Segmentation (2017, critical): “Compared to more recent works [24, 4, 25, 5], our method falls behind”
Fully Convolutional Instance-Aware Semantic Segmentation (2016, background): “There are some endeavors [22, 26] trying to extend FCNs for instance-aware semantic segmentation, by grouping/clustering the FCN’s output.”
Recurrent Instance Segmentation (2015, neutral): “The work in [15] builds an instance segmentation model standing on these previous approaches.”
Pixelwise Instance Segmentation with a Dynamically Instantiated Network (2017, methodology): “Other segmentation-based approaches, [3, 28] are not fully endto-end trained.”
Shape-aware Instance Segmentation (2016, background): “For instance, the Proposal-free Network (PFN) of [20] predicts the number of instances and, at each pixel, a semantic label and the location of its enclosing bounding box.”
InstanceCut: From Edges to Instances with MultiCut (2016, background): “The work [34] presents a proposal-free network that produces an instance-agnostic semantic segmentation, number of instances for the image, and a per-pixel bounding box of the corresponding instance.”
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (2016, methodology): “In another line of research, [49], [73] pursue instance segmentation, jointly tackling object detection and semantic segmentation.”
3D Instance Segmentation of MVS Buildings (2021, methodology): “These methods work in a bottomup manner, generate embedding features [23], [24] for each pixel and use post-processing methods such as clustering [25], [26] or graph theory [27] to classify these pixels.”
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics (2017, methodology): “We use a regression approach for instance segmentation [29].”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, background): “We note that there are several instance segmentation works [77, 68, 76, 2, 47, 35, 55, 20, 17, 43, 37, 30, 50, 53, 6], which could be potentially extended to bottom-up panoptic segmentation.”
Pixel-Level Encoding and Depth Layering for Instance-Level Semantic Labeling (2016, methodology): “Other methods score region proposals for instance segmentation [7,14] or object detection [11,12,23,24], or use a multi-stage neural network for these tasks [8,19].”
Bridging Category-level and Instance-level Semantic Image Segmentation (2016, background): “One notable work is the proposal-free network (PFN) [12], which for all pixels predicts semantic scores and instance bounding-boxes, as well as category-wise numbers of instances per image, based on which a clustering algorithm is applied for post-processing to locate instances.”
Mask Scoring R-CNN (2019, methodology): “[24] used spectral clustering to cluster the pixels.”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, methodology): “Proposal-free network [45] applies spectral clustering to group segmentation results from DeepLab [10], while Zhang et al.”
Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected MRFs (2015, methodology): “[19] trained a multi-output CNN that jointly predicts pixel-level class labeling of the image as well as bounding box locations and object instance numbers.”
Reversible Recursive Instance-Level Object Segmentation (2015, methodology): “When evaluated with the annotations from VOC 2012 dataset, Table 2 and Table 3 present the comparison of the proposed R2-IOS with three state-of-the-art methods [10][17][15] using AP r metric at IoU score 0.”
Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs (2019, methodology): “However, the CoordConv technique had been used beforehand [Zhao et al., 2015, Liang et al., 2015, Watters et al., 2017, Wojna et al., 2017, Perez et al., 2017, Nash et al., 2017, Ulyanov et al., 2017] and its origin is unclear.”
Attention-Guided Unified Network for Panoptic Segmentation (2018, methodology): “Moreover, segmentation-based methods aggregated pixel-level cues to compose instances combined with semantic segmentation [2, 26, 32] or depth ordering [44] results.”