Reversible Recursive Instance-Level Object Segmentation

By Xiaodan Liang, Yunchao Wei, Xiaohui Shen, Zequn Jie, Jiashi Feng, Liang Lin, Shuicheng Yan. 2015

61 citations recorded by the source corpus.

In this work, we propose a novel Reversible Recursive Instance-level Object Segmentation (R2-IOS) framework to address the challenging instance-level object segmentation task. R2-IOS consists of a reversible proposal refinement sub-network that predicts bounding box offsets for refining the object proposal locations, and an instance-level segmentation sub-network that generates the foreground mask of the dominant object instance in each proposal. By being recursive, R2-IOS iteratively optimizes the two subnetworks during joint training, in which the refined object proposals and improved segmentation predictions are alternately fed into each other to progressively increase the network capabilities. By being reversible, the proposal refinement sub-network adaptively determines an optimal number of refinement iterations required for each proposal during both training and testing. Furthermore, to handle multiple overlapped instances within a proposal, an instance-aware denoising autoencoder is introduced into the segmentation sub-network to distinguish the dominant object from other distracting instances. Extensive experiments on the challenging PASCAL VOC 2012 benchmark well demonstrate the superiority of R2-IOS over other state-of-the-art methods. In particular, the APr over 20 classes at 0:5 IoU achieves 66:7%, which significantly outperforms the results of 58:7% by PFN [17] and 46:3% by [22].

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Recurrent Neural Networks for Semantic Instance Segmentation (2017, methodology): “Compared to more recent works [24, 4, 25, 5], our method falls behind”
Pixelwise Instance Segmentation with a Dynamically Instantiated Network (2017, methodology): “Thereafter, each bounding box is refined into a segmentation [18, 19, 27, 32, 26].”
Deep Watershed Transform for Instance Segmentation (2016, methodology): “Proposal + recursion: [14] proposed a novel method that recursively refines proposals.”
InstanceCut: From Edges to Instances with MultiCut (2016, methodology): “Several methods produce proposals for instance segmentations and combine them, based on learned scores [33, 41, 42] or generate parts of instances and then combine them [14, 38].”
Path Aggregation Network for Instance Segmentation (2018, methodology): “Motivation Fully-connected layers, or MLP, were widely used in mask prediction in instance segmentation [10, 41, 34] and mask proposal generation [48, 49].”
Recurrent Pixel Embedding for Instance Grouping (2017, methodology): “Common approaches to instance segmentation ﬁrst generate region proposals or class-agnostic bounding boxes, segment the foreground objects within each proposal and classify the objects in the bounding box [92, 53, 31, 12, 17, 56, 34].”
Bottom-up Instance Segmentation using Deep Higher-Order CRFs (2016, background): “[23] address some of these shortcomings by iteratively reﬁning initial object proposals, whilst [7] generates box-proposals, creates masks from these proposals and then classiﬁes these masks in a single network trained with three loss functions summed together.”
Instance-level Human Parsing via Part Grouping Network (2018, methodology): “The very recent work [16] explored this task following the “parsing-by-detection” pipeline [12,21,7,31,13] that ﬁrstly localizes bounding boxes of instances and then performs ﬁne-grained semantic parsing for each box.”
Semantic Instance Segmentation via Deep Metric Learning (2017, methodology): “A common approach to this problem (e.g., [10, 4, 16, 6]) is ﬁrst to use some mechanism to predict object bounding boxes (e.g., by running a class-level object detector, or by using a class agnostic box proposal method such as EdgeBoxes), and then to run segmentation and classiﬁcation within each…”
Semantic Segmentation With Context Encoding and Multi-Path Decoding (2020, methodology): “some limitations when directly applying DCNN to the dense prediction tasks like semantic segmentation [9], [17], [26], [38], [62], [72], [82].”
Semantic Object Parsing with Local-Global Long Short-Term Memory (2015, background): “Recently, many research works [15][33][31][19][20][15] have been devoted to exploring various Convolutional Neural Networks (CNN) based models for semantic object parsing, due to their excellent performance in image classification [29], object segmentation [18][17] and object detection [16].”
SGN: Sequential Grouping Networks for Instance Segmentation (2017, methodology): “A recursive process was proposed in [22] to iteratively refine the predicted pixel-wise masks.”
Deep GrabCut for Object Selection (2017, methodology): “The works of [5, 13, 14] are most relevant to ours as they use neural nets.”
Semi-convolutional Operators for Instance Segmentation (2018, methodology): “SDS [20] PFN [33] DIN [3] MNC [10] FCIS [31] R2-IOS [32] DML [15] R.”
Attentive Contexts for Object Detection (2016, background): “Many recent works [9], [17], [18], [38] have also demonstrated the effectiveness of this pipeline.”