Learning Instance Occlusion for Panoptic Segmentation

By Justin Lazarow, Kwonjoon Lee, Zhuowen Tu. 2019

78 citations recorded by the source corpus.

Panoptic segmentation requires segments of both “things” (countable object instances) and “stuff” (uncountable and amorphous regions) within a single output. A common approach involves the fusion of instance segmentation (for “things”) and semantic segmentation (for “stuff”) into a non-overlapping placement of segments, and resolves overlaps. However, instance ordering with detection confidence do not correlate well with natural occlusion relationship. To resolve this issue, we propose a branch that is tasked with modeling how two instance masks should overlap one another as a binary relation. Our method, named OCFusion, is lightweight but particularly effective in the instance fusion process. OCFusion is trained with the ground truth relation derived automatically from the existing dataset annotations. We obtain state-of-the-art results on COCO and show competitive results on the Cityscapes panoptic segmentation benchmark.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Occluded Video Instance Segmentation: A Benchmark (2021, background): “OCFusion (Lazarow et al., 2020) introduces the occlusion head to indicate the occlusion relation between each pair of mask proposals.”
Image Segmentation Using Deep Learning: A Survey (2020, neutral): “5 OCFusion∗ [174] ResNet-50 41.”
Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers (2021, methodology): “Recently, OCFusion [32] uses an additional branch to model instances fusion process for replacing detection confidence in panoptic segmentation.”
Video Panoptic Segmentation (2020, background): “Since then, a large number of works [7,8,10,15,17–20,24, 31,37,40] have proposed learning-based approaches to this new benchmark task, conﬁrming its importance to the ﬁeld.”
Self-Supervised Scene De-Occlusion (2020, background): “Another stream of works on panoptic segmentation [26, 27] design end-to-end training procedures to resolve overlapping segments.”
Open-Vocabulary Panoptic Segmentation with MaskCLIP (2022, methodology): “Existing panoptic segmentation methods [16, 15, 21, 33, 17] and instance segmentation approaches [9] deal with a ﬁxed set of category deﬁnitions, which are essentially represented by categorical labels without semantic relations.”
Scaling Wide Residual Networks for Panoptic Segmentation (2020, methodology): “Top-down approaches [43, 67, 50, 54, 60, 94, 51, 10, 47, 91] typically pair Mask R-CNN [27] with a light-weight ‘stuff‘ segmentation branch, while bottom-up approaches [95, 20, 87, 12, 88] group ‘thing‘ pixels from semantic segmentation predictions.”