Fully Convolutional Instance-Aware Semantic Segmentation

By Yi Li, Haozhi Qi, Jifeng Dai, Xiangyang Ji, Yichen Wei. 2016

1,052 citations recorded by the source corpus.

We present the first fully convolutional end-to-end solution for instance-aware semantic segmentation task. It inherits all the merits of FCNs for semantic segmentation [29] and instance mask proposal [5]. It performs instance mask prediction and classification jointly. The underlying convolutional representation is fully shared between the two sub-tasks, as well as between all regions of interest. The network architecture is highly integrated and efficient. It achieves state-of-the-art performance in both accuracy and efficiency. It wins the COCO 2016 segmentation competition by a large margin. Code would be released at https://github.com/daijifeng001/TA-FCN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

End-to-End Object Detection with Transformers (2020, methodology): “Following [21], we also increase the feature resolution by adding a dilation to the last stage of the backbone and removing a stride from the first convolution of this stage.”
Mask R-CNN (2017, background): “[26] combined the segment proposal system in [8] and object detection system in [11] for “fully convolutional instance segmentation” (FCIS).”
Relation Networks for Object Detection (2017, background): “We do not see any reason preventing it from finding broader applications in vision tasks, such as instance segmentation [30], action recognition [41], object relationship detection [28], caption [50], VQA [1], etc.”
Objects as Points (2019, background): “Object detection powers many vision tasks like instance segmentation [7, 21, 32], pose estimation [3, 15, 39], tracking [24,27], and action recognition [5].”
UPSNet: A Unified Panoptic Segmentation Network (2019, methodology): “[21] proposed a fully convolutional instance-aware segmentation method, where instance mask proposals [7] are married with fully convolutional networks [26].”
Panoptic Feature Pyramid Networks (2019, background): “Another interesting direction is to use position-sensitive pixel labeling [35] to encode instance information fully convolutionally; [46, 9] build on this.”
Panoptic Segmentation (2018, background): “By framing this task and providing a high-quality dataset, COCO helped define a new and exciting research direction and led to many recent breakthroughs in instance segmentation [36, 24, 14].”
Recurrent Neural Networks for Semantic Instance Segmentation (2017, methodology): “In this paper, we argue that both ranking thousands of object-prominent regions [17, 18, 24, 23, 11, 19] or decomposing the problem into two independent modules [6, 12, 34] are unnatural approaches that are far away from the sequential exploration that we as humans follow to solve related tasks such”
Pix2seq: A Language Modeling Framework for Object Detection (2021, methodology): “Per (Carion et al., 2020), we also experiment with the DC5 variant of ResNet (Li et al., 2017), which increases the resolution of its output feature map by a factor of two.”
3D Instance Segmentation of MVS Buildings (2021, methodology): “Inspired by full convolutional instance-aware semantic segmentation (FCIS) [29] and You Only Look At Coefﬁcients (YOLACT) [30], BlendMask uses a blender module to merge top-level coarse instance information with lower-level ﬁne granularities [31].”
K-Net: Towards Unified Image Segmentation (2021, neutral): “‘Top-down’ approaches [2, 15, 22, 27, 33] first detect accurate”
Hybrid Task Cascade for Instance Segmentation (2019, background): “In a recent work, FCIS [23] extends InstanceFCN and presents a fully convolutional approach for instance segmentation.”
Path Aggregation Network for Instance Segmentation (2018, methodology): “With ResNet-50 [23] as the initial network, our PANet tested with a single scale already outperforms champion of COCO 2016 Challenge in both object detection [27] and instance segmentation [33] tasks.”
Class-Incremental Instance Segmentation via Multi-Teacher Networks (2021, methodology): “Two-stage methods formulate this task as the paradigm of “Detect then Segment” (Li et al. 2017; He et al. 2017; Liu et al. 2018; Chen et al. 2019).”
Taskonomy: Disentangling Task Transfer Learning (2018, methodology): “We distill knowledge from state-of-the-art models [10, 20] trained on ImageNet [33] and MS-COCO [22] by using them to annotate our dataset and then supervising models with those annotations.”
Mask Scoring R-CNN (2019, background): “FCIS [23] takes position-sensitive maps with inside/outside scores to generate the instance segmentation results.”
YOLACT: Real-Time Instance Segmentation (2019, methodology): “Mask Quality Because we produce a final mask of size 138 ⇥ 138, and because we create masks directly from the original features (with no repooling to transform and potentially misalign the features), our masks for large objects are noticeably higher quality than those of Mask R-CNN [18] and FCIS [24”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, methodology): “FCIS [44], the first Fully Convolutional Network (FCN) [53] for instance segmentation, enriches the position-sensitive score maps from [16] by further considering inside/outside score maps.”
Attention-Guided Unified Network for Panoptic Segmentation (2018, background): “Recently, the rapid progress in object detection [13, 14, 34] and instance segmentation [9, 15, 25, 31] made it possible to achieve object localization and segmentation at a finer level.”