Instance-Sensitive Fully Convolutional Networks

By Jifeng Dai, Kaiming He, Yi Li, Shaoqing Ren. 2016

408 citations recorded by the source corpus.

Fully convolutional networks (FCNs) have been proven very successful for semantic segmentation, but the FCN outputs are unaware of object instances. In this paper, we develop FCNs that are capable of proposing instance-level segment candidates. In contrast to the previous FCN that generates one score map, our FCN is designed to compute a small set of instance-sensitive score maps, each of which is the outcome of a pixel-wise classifier of a relative position to instances. On top of these instance-sensitive score maps, a simple assembling module is able to output instance candidate at each position. In contrast to the recent DeepMask method for segmenting instances, our method does not have any high-dimensional layer related to the mask resolution, but instead exploits image local coherence for estimating instances. We present competitive results of instance segment proposal on both PASCAL VOC and MS COCO.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Mask R-CNN (2017, methodology): “DeepMask [32] and following works [33, 8] learn to propose segment candidates, which are then classified by Fast R-CNN.”
Feature Pyramid Networks for Object Detection (2016, background): “Existing mask proposal methods [27, 28, 4] are based on densely sampled image pyramids ( e.g ., scaled by 2 {− 2:0 .”
UPSNet: A Unified Panoptic Segmentation Network (2019, methodology): “[21] proposed a fully convolutional instance-aware segmentation method, where instance mask proposals [7] are married with fully convolutional networks [26].”
Recurrent Neural Networks for Semantic Instance Segmentation (2017, methodology): “[11, 10] propose multi-task network cascades where box coordinates, binary masks and class scores are predicted in chain, allowing the network to reuse information learned in previous stages.”
Fully Convolutional Instance-Aware Semantic Segmentation (2016, methodology): “As shown in [5], the approach is state-of-the-art for the object mask proposal task.”
Pixelwise Instance Segmentation with a Dynamically Instantiated Network (2017, background): “Another related approach [11, 49] is to use segment-based region proposals [9, 35, 36] instead of box-based proposals.”
Shape-aware Instance Segmentation (2016, methodology): “These proposals can be learned [13, 19, 7] or sampled by sliding windows [23, 5], and greatly facilitate the task of identifying the different instances, may they be from the same category or different ones.”
InstanceCut: From Edges to Instances with MultiCut (2016, methodology): “Several methods produce proposals for instance segmentations and combine them, based on learned scores [33, 41, 42] or generate parts of instances and then combine them [14, 38].”
R-FCN: Object Detection via Region-based Fully Convolutional Networks (2016, background): “The concept of position-sensitive score maps is partially inspired by [3] that develops FCNs for instance-level semantic segmentation.”
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs (2016, methodology): “Interestingly, the atrous convolution technique has also been adopted for a broader set of tasks, such as object detection [12], [77], instance-level segmentation [78], visual question answering [79], and optical flow [80].”
Hybrid Task Cascade for Instance Segmentation (2019, methodology): “In a recent work, FCIS [23] extends InstanceFCN and presents a fully convolutional approach for instance segmentation.”
Path Aggregation Network for Instance Segmentation (2018, background): “Results of [8, 33] show that FCN is also competent in predicting pixelwise masks for instances.”
Class-Incremental Instance Segmentation via Multi-Teacher Networks (2021, methodology): “One-stage methods (Dai et al. 2016; Liu et al. 2018; Bolya et al. 2019; Chen et al. 2018) are conceptually faster than the two-stage methods.”
Attend Refine Repeat: Active Box Proposal Generation via In-Out Localization (2016, background): “Several approaches have been proposed in the literature for this task [1, 2, 5, 6, 8, 18, 25, 26, 29, 40, 44].”
Mask Scoring R-CNN (2019, background): “[6] proposed instance-sensitive FCNs to generate the position-sensitive maps and assembled them to obtain the ﬁnal masks.”
Meta R-CNN: Towards General Solver for Instance-Level Low-Shot Learning (2019, methodology): “Proposalbased methods [30, 31, 7, 6] predict object masks based on the generated region proposals while image-based methods [47, 48, 44, 2] produce a pixel-level segmentation map over the image to identify object instance.”
YOLACT: Real-Time Instance Segmentation (2019, methodology): “One-stage instance segmentation methods generate position sensitive maps that are assembled into final masks with position-sensitive pooling [6, 24] or combine semantic segmentation logits and direction prediction logits [4].”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, methodology): “FCIS [44], the first Fully Convolutional Network (FCN) [53] for instance segmentation, enriches the position-sensitive score maps from [16] by further considering inside/outside score maps.”
InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting (2019, background): “Recently, FCIS [28] proposed the ﬁrst fully convolutional end-to-end solution to instance segmentation, which predicted position-sensitive channels [11] for instance segmentation.”