SSAP: Single-Shot Instance Segmentation With Affinity Pyramid

By Naiyu Gao, Yanhu Shan, Yupei Wang, Xin Zhao, Yinan Yu, Ming Yang, Kaiqi Huang. 2019

248 citations recorded by the source corpus.

Recently, proposal-free instance segmentation has received increasing attention due to its concise and efficient pipeline. Generally, proposal-free methods generate instance-agnostic semantic segmentation labels and instance-aware features to group pixels into different object instances. However, previous methods mostly employ separate modules for these two sub-tasks and require multiple passes for inference. We argue that treating these two sub-tasks separately is suboptimal. In fact, employing multiple separate modules significantly reduces the potential for application. The mutual benefits between the two complementary sub-tasks are also unexplored. To this end, this work proposes a single-shot proposal-free instance segmentation method that requires only one single pass for prediction. Our method is based on a pixel-pair affinity pyramid, which computes the probability that two pixels belong to the same instance in a hierarchical manner. The affinity pyramid can also be jointly learned with the semantic class labeling and achieve mutual benefits. Moreover, incorporating with the learned affinity pyramid, a novel cascaded graph partition module is presented to sequentially generate instances from coarse to fine. Unlike previous time-consuming graph partition methods, this module achieves 5× speedup and 9% relative improvement on Average-Precision (AP). Our approach achieves new state of the art on the challenging Cityscapes dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity (2022, background): “The pixel-level pairwise affinity can be either hand-constructed [7, 23, 24] or learned [16, 17, 28, 34, 35,40,50].”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, methodology): “Then, instance segments are obtained by grouping ‘thing’ pixels with various methods, such as instance center regression [44, 86, 70, 100, 20], Watershed transform [88, 3, 8], Hough-voting [4, 53, 8], or pixel affinity [45, 66, 81, 30, 8].”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, methodology): “For example, DWT [2] and SSAP [23] use an average of semantic segmentation scores for some easy classes and use random scores for other harder classes.”
Unifying Training and Inference for Panoptic Segmentation (2020, neutral): “2 – – – – – – – – SSAP [7] R-50 36.”
Generalized Few-shot Semantic Segmentation (2020, background): “Few-Shot Segmentation (FS-Seg) places semantic segmentation in the few-shot scenario [31, 33, 25, 28, 48, 17, 45, 12, 9, 56, 2, 34, 46, 23, 37, 11, 24, 38, 43, 40, 20, 52], where dense pixel labeling is performed on new classes with only a few support samples.”
Deep Snake for Real-Time Instance Segmentation (2020, unknown): “There exist some pixel-based methods [2, 31, 28, 12, 43] that are free of region proposals.”
PolyTransform: Deep Polygon Transformer for Instance Segmentation (2019, background): “1 SSAP [18] fine 37.”
SOLOv2: Dynamic and Fast Instance Segmentation (2020, methodology): “Bottom-up methods [27, 9, 24, 10] view the task as a label-then-cluster problem, e . g ., learning the per-pixel embeddings and then clustering them into groups.”
Towards Bounding-Box Free Panoptic Segmentation (2020, methodology): “In [12], graph partioning is performed on an affinity pyramid computed within a fixed window for each pixel.”
Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers (2021, background): “The panoptic segmentation literature mainly treats this problem as a joint task of instance segmentation and semantic segmentation where things and stuff are handled separately [18, 19].”
k-means Mask Transformer (2022, methodology): “…clustering-based methods usually operate on semantic segments [13,15,18] and group ‘thing’ pixels into instance segments with various representations, such as instance center regression [50,89,77,104,22,94,63], Watershed transform [91,5], Hough-voting [6,59,92], or pixel aﬃnity [51,69,85,36,47].”
SOLQ: Segmenting Objects by Learning Queries (2021, methodology): “Bottom-up methods [25, 26, 27, 28] learn per-pixel embeddings via semantic label and then cluster them into instance groups.”
CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation (2022, methodology): “Other proxy-based methods typically start with semantic segments [11,13,16] and group ‘thing’ pixels into instance segments with various proxy tasks, such as instance center regression [19,42,56,67,79,83,90], Watershed transform [4, 8, 81], Hough-voting [5, 8, 51], or pixel affinity [8, 29, 43, 61, ”
EfficientPS: Efficient Panoptic Segmentation (2020, methodology): “We compare the performance of our proposed EfficientPS against state-of-the-art models on the Cityscapes dataset including WeaklySupervised (Li et al, 2018b), TASCNet (Li et al, 2018a), Panoptic FPN (Kirillov et al, 2019), AUNet (Li et al, 2019b), UPSNet (Xiong et al, 2019), DeeperLab (Yang et al, 2”
Fully Convolutional Networks for Panoptic Segmentation (2020, methodology): “Although attempt [51, 10, 6] has been made to predict things without boxes, extra predictions (e.”