An End-To-End Network for Panoptic Segmentation

By Huanyu Liu, Chao Peng, Changqian Yu, Jingbo Wang, X. Liu, Gang Yu, Wei Jiang. 2019

150 citations recorded by the source corpus.

Panoptic segmentation, which needs to assign a category label to each pixel and segment each object instance simultaneously, is a challenging topic. Traditionally, the existing approaches utilize two independent models without sharing features, which makes the pipeline inefficient to implement. In addition, a heuristic method is usually employed to merge the results. However, the overlapping relationship between object instances is difficult to determine without sufficient context information during the merging process. To address the problems, we propose a novel end-to-end Occlusion Aware Network (OANet) for panoptic segmentation, which can efficiently and effectively predict both the instance and stuff segmentation in a single network. Moreover, we introduce a novel spatial ranking module to deal with the occlusion problem between the predicted instances. Extensive experiments have been done to validate the performance of our proposed method and promising results have been achieved on the COCO Panoptic benchmark.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, neutral): “[59] adopt a Spatial Ranking module.”
Panoptic Segmentation (2018, neutral): “[23, 47, 48, 27, 22, 21, 17] address PS).”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, methodology): “Then, the instance segments (‘thing’) and semantic segments (‘stuff’) [13] are fused by merging modules [54, 56, 74, 67, 101] to generate panoptic segmentation.”
Object-Contextual Representations for Semantic Segmentation (2019, neutral): “In general, the performance of “Panoptic-FPN + OCR” is very competitive compared to various recent methods [66,44,69].”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, background): “[52] propose the Spatial Ranking module to resolve the overlapping instance masks.”
Attention-Guided Unified Network for Panoptic Segmentation (2018, neutral): “Most recently works include [22, 29, 40].”
Image Segmentation Using Deep Learning: A Survey (2020, neutral): “7 OANet [172] ResNet-50 39.”
Unifying Training and Inference for Panoptic Segmentation (2020, methodology): “proposes to directly learn the ordering of “thing” instances for conflict resolution [20].”
EfficientPS: Efficient Panoptic Segmentation (2020, methodology): “Whereas, Liu et al (2019) address the problem by introducing the spatial ranking module and Li et al (2018a) propose a method that learns a binary mask to constrain output distributions of ‘stuff’ and ‘thing’ explicitly.”
Seamless Scene Segmentation (2019, background): “Independently and simultaneously to our paper, a number of works [23, 29, 35, 54, 55] have proposed panoptic segmentation provided by a single deep network, confirming the importance of this task to the field.”
k-means Mask Transformer (2022, methodology): “…box-based methods [54] that decompose panoptic segmentation into multiple surrogate tasks ( e.g ., predicting masks for each detected object bounding box [40], followed by fusing the instance segments (‘thing’) and semantic segments (‘stuﬀ’) [14] with merging modules [62,79,67,105,101,60]).”
Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey (2020, methodology): “pixel binary classication loss is used. Focal loss is used to handle foreground background class imbalance. segmentation models on those datasets. 5. Panoptic Segmentation Panoptic segmentation (PS) [40, 165, 99, 166, 167, 168] is the combination of semantic segmentation and instance segmentation. T”
Learning Instance Occlusion for Panoptic Segmentation (2019, methodology): “Since our occlusion head can deal with two arbitrary masks, it offers more flexibility over these approaches which attempt to “rerank” the masks in a linear fashion [38, 23].”
Video Panoptic Segmentation (2020, background): “[24] design a spatial ranking module to address the occlusion between the predicted instances.”
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2020, methodology): “[149] proposed OANet that also introduced the FPN based on the mask-RCNN.”