Weakly- and Semi-Supervised Panoptic Segmentation

By Qizhu Li, Anurag Arnab, Philip H. S. Torr. 2018

169 citations recorded by the source corpus.

We present a weakly supervised model that jointly performs both semantic- and instance-segmentation – a particularly relevant problem given the substantial cost of obtaining pixel-perfect annotation for these tasks. In contrast to many popular instance segmentation approaches based on object detectors, our method does not predict any overlapping instances. Moreover, we are able to segment both “thing” and “stuff” classes, and thus explain all the pixels in the image. “Thing” classes are weakly-supervised with bounding boxes, and “stuff” with image-level tags. We obtain state-of-the-art results on Pascal VOC, for both full and weak supervision (which achieves about 95% of fully-supervised performance). Furthermore, we present the first weakly-supervised results on Cityscapes for both semantic- and instance-segmentation. Finally, we use our weakly supervised framework to analyse the relationship between annotation quality and predictive performance, which is of interest to dataset creators.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

UPSNet: A Unified Panoptic Segmentation Network (2019, methodology): “via a PSPNet, and for instance segmentation utilizing a Mask R-CNN [15], followed by simple heuristic decisions to produce a single void, stuff, or thing instance label per pixel. Recently, Li et al. [19] introduced a weakly- and semi-supervised panoptic segmentation method where they relieve some o”
Panoptic Feature Pyramid Networks (2019, neutral): “Instead, we compare on val to the recent work of Arnab and Torr [1, 34] who develop a novel approach for panoptic segmentation, named DIN.”
Panoptic Segmentation (2018, neutral): “[23, 48, 49, 27, 22, 21, 17] address PS).”
WoodScape: A Multi-Task, Multi-Camera Fisheye Dataset for Autonomous Driving (2019, methodology): “The dataset also provides instance segmentation labels to explore panoptic segmentation models [34].”
Learning to Fuse Things and Stuff (2018, methodology): “[18], and the panoptic segmentation performance of our uniﬁed TASC-Net outperforms their fully supervised joint solution by a large margin.”
Attention-Guided Unified Network for Panoptic Segmentation (2018, methodology): “Overall, the proposed method surpass previous stateof-the-art [24], with a 5.”
SSAP: Single-Shot Instance Segmentation With Affinity Pyramid (2019, methodology): “Another work [2, 32] combines the predictions of detection and semantic segmentation with a CRFasRNN [50] to generate instances.”
AdaptIS: Adaptive Instance Selection Network (2019, neutral): “3 CRF + PSPNet [18] ImageNet ResNet-101 53.”
An End-To-End Network for Panoptic Segmentation (2019, methodology): “A weakly supervised model that jointly performs semantic and instance segmentation was proposed by [22].”
EfficientPS: Efficient Panoptic Segmentation (2020, methodology): “In order to mitigate this problem, Li et al (2018b) propose a weakly supervised model where ‘thing’ classes are weakly supervised by bounding boxes and ‘stuff’ classes are supervised with image-level tags.”
Seamless Scene Segmentation (2019, background): “The supervised variant in [30] generates panop-tic segmentation results but i) requires separate (external) input for bounding box proposals and ii) exploits a CRF during inference, increasing the complexity of the model.”
Mixed supervision for surface-defect detection: from weakly to fully supervised learning (2021, unknown): “Weakly supervised learning Various weakly supervised deep-learning approaches have been developed in the context of semantic segmentation and object detection [26, 29, 2, 16, 37, 36].”
SpatialFlow: Bridging All Tasks for Panoptic Segmentation (2019, methodology): “…and stuff segmentation; Li et al. (Li et al. 2018a) proposed a uniﬁed network named TASCNet by considering things and stuff consistency; Li et al. (Li et al. 2018b) proposed AUNet which utilized the attention module to capture the complementary information delivered by things and stuff;…”
Fully Convolutional Networks for Panoptic Segmentation (2020, methodology): “Although methods of [23, 10] represent things and stuff uniformly, the inherent ambiguity cannot be resolved well merely with the pixel-level affinity, which yields the performance drop in complex scenarios.”