Iterative Instance Segmentation

By Ke Li, Bharath Hariharan, Jitendra Malik. 2015

153 citations recorded by the source corpus.

Existing methods for pixel-wise labelling tasks generally disregard the underlying structure of labellings, often leading to predictions that are visually implausible. While incorporating structure into the model should improve prediction quality, doing so is challenging - manually specifying the form of structural constraints may be impractical and inference often becomes intractable even if structural constraints are given. We sidestep this problem by reducing structured prediction to a sequence of unconstrained prediction problems and demonstrate that this approach is capable of automatically discovering priors on shape, contiguity of region predictions and smoothness of region contours from data without any a priori specification. On the instance segmentation task, this method outperforms the state-of-the-art, achieving a mean APr of 63:6% at 50% overlap and 43:3% at 70% overlap.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Fully Convolutional Instance-Aware Semantic Segmentation (2016, methodology): “This paradigm includes most state-of-the-art approaches, such as SDS [15], Hypercolumn [16], CFM [7], MNC [8], MultiPathNet [42], and iterative approach [21].”
Pixelwise Instance Segmentation with a Dynamically Instantiated Network (2017, methodology): “We do not require any post-processing, such as the patch aggregation of [32], “mask-voting” of [11], “superpixel projection” of [18, 19, 26] or spectral clustering of [28].”
Beyond Skip Connections: Top-Down Modulation for Object Detection (2016, background): “Contemporary works [5, 12, 32, 43], which incorporate top-down feedback loop in ConvNets, have used ‘unrolling’ to train these networks stagewise.”
Shape-aware Instance Segmentation (2016, methodology): “This framework was improved by the development of Hypercolumn features [14] and the use of a fully convolutional network (FCN) to encode category-speciﬁc shape priors [19].”
Deep Watershed Transform for Instance Segmentation (2016, background): “Additionally, [13] proposed deep convolutional neural network that learns the underlying shapes of objects, and performs multiple unconstrained inference steps to refine regions corresponding to an object instance while ignoring neighboring pixels that do not belong to the primary object.”
Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency (2017, methodology): “training data, we also use the Imagenet [27] objects from PASCAL 3D (using an off-the shelf instance segmentation method [22] to compute foreground masks on these).”
Object-Contextual Representations for Semantic Segmentation (2019, background): “Various coarse-to-fine segmentation schemes have been developed [15,18,33,59,27,32,84] to gradually refine the segmentation maps from coarse to fine.”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, neutral): “[43] iteratively apply the prediction.”
Boundary-Aware Instance Segmentation (2016, methodology): “These proposals can be learned [14, 21, 8] or sampled by sliding windows [25, 6], and greatly facilitate the task of identifying the different instances, may they be from the same category or different ones.”
BiSeg: Simultaneous Instance Segmentation and Semantic Segmentation with Fully Convolutional Networks (2017, methodology): “We compare BiSeg with recent methods [2, 7, 8, 9, 16, 17, 19, 20, 23], and variants of our own method with different settings.”
SegFix: Model-Agnostic Boundary Refinement for Segmentation (2020, background): “Refinement for Segmentation: Extensive studies [21,23,27,35,36] have proposed various mechanisms to refine the segmentation maps from coarse to fine.”
DeMoN: Depth and Motion Network for Learning Monocular Stereo (2016, background): “[23], who train each iteration in a separate step and therefore need to store predictions as input for the next stage.”
Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers (2021, methodology): “Li and Malik [35] first propose a method by extending [34], which iteratively enlarges the modal bounding box following the direction of high heatmap values and synthetically adds occlusion.”
Annotating Object Instances with a Polygon-RNN (2017, methodology): “With new benchmarks such as Cityscapes [6], object instance segmentation is also gaining steam [14, 24, 34, 21, 29].”
End-to-End Instance Segmentation with Recurrent Attention (2016, methodology): “Simultaneous detection and segmentation (SDS) methods [17, 18, 24, 23] apply newer CNN-based detectors such as RCNN [14], and perform segmentation using the bounding boxes as starting point.”
Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++ (2018, background): “Most approaches to object instance segmentation [16, 29, 39, 37, 21, 22, 12, 1, 18] operate on the pixel-level.”
An application of cascaded 3D fully convolutional networks for medical image segmentation (2018, background): “In effect, this step narrows down and simplifies the search space for the FCN to decide which voxels belong to the background or any of the foreground classes; this strategy has been successful in many computer vision problems (Viola and Jones, 2004; Li et al., 2016).”
Contextual Priming and Feedback for Faster R-CNN (2016, methodology): “Given that segmentation is used as feedback, it is supposed to self-improve across units, therefore we use the Stage-2 output as our final output (similar to [49, 50]).”
Video Object Segmentation without Temporal Information (2017, methodology): “approaches based on iteration [34], and recurrent neural networks [58].”