Affinity Derivation and Graph Merge for Instance Segmentation

By Yiding Liu, Siyu Yang, Bin Li, Wen-gang Zhou, Jizheng Xu, Houqiang Li, Yan Lu. 2018

111 citations recorded by the source corpus.

We present an instance segmentation scheme based on pixel affinity information, which is the relationship of two pixels belonging to the same instance. In our scheme, we use two neural networks with similar structures. One predicts the pixel level semantic score and the other is designed to derive pixel affinities. Regarding pixels as the vertexes and affinities as edges, we then propose a simple yet effective graph merge algorithm to cluster pixels into instances. Experiments show that our scheme generates fine grained instance masks. With Cityscape training data, the proposed scheme achieves 27.3 AP on test set.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, methodology): “SSAP [27] exploits the pixel-pair aﬃnity pyramid [58] enabled by an eﬃcient graph partition method [42].”
Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity (2022, background): “The pixel-level pairwise affinity can be either hand-constructed [7, 23, 24] or learned [16, 17, 28, 34, 35,40,50].”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, methodology): “Then, instance segments are obtained by grouping ‘thing’ pixels with various methods, such as instance center regression [44, 86, 70, 100, 20], Watershed transform [88, 3, 8], Hough-voting [4, 53, 8], or pixel affinity [45, 66, 81, 30, 8].”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, methodology): “We note that there are several instance segmentation works [77, 68, 76, 2, 48, 35, 56, 20, 18, 44, 37, 30, 51, 54, 6], which could be potentially extended to bottom-up panoptic segmentation.”
SSAP: Single-Shot Instance Segmentation With Affinity Pyramid (2019, methodology): “[40] employ two models and require multiple passes for the RoIs generated from semantic segmentation results.”
DeeperLab: Single-Shot Image Parser (2019, methodology): “On the other hand, the bottom-up approaches [73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84] generally adopt a two-stage processing: pixel-level predictions produced by the segmentation module are clustered together to form the instancelevel predictions.”
PolyTransform: Deep Polygon Transformer for Instance Segmentation (2019, unknown): “1 GMIS [43] fine+coarse − 27.”
CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation (2022, methodology): “Other proxy-based methods typically start with semantic segments [11,13,16] and group ‘thing’ pixels into instance segments with various proxy tasks, such as instance center regression [19,42,56,67,79,83,90], Watershed transform [4, 8, 81], Hough-voting [5, 8, 51], or pixel affinity [8, 29, 43, 61, ”
Deep Snake for Real-Time Instance Segmentation (2020, unknown): “There exist some pixel-based methods [2, 31, 28, 12, 43] that are free of region proposals.”
MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation (2019, methodology): “We address this issue by predicting only similarity between neighboring points at multiple scales, as did in [11] for 2D instance segmentation, and develops a clustering algorithm to group points based on the learned local similarity and the input mesh topology.”
BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation (2020, background): “Recognizing these difficulties, a line of research takes a bottom-up strategy [1, 19, 20].”
OccuSeg: Occupancy-Aware 3D Instance Segmentation (2020, methodology): “A popular solution for 3D instance segmentation [26, 41, 21] is to marry the powerful 3D feature extractors (spatially sparse convolutional networks [13] or PointNet++ [37]) with conventional 2D image instance segmentation techniques [16, 7, 27].”
ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors (2019, background): “Existing instance segmentation algorithms can be categorized into two major approaches: detection-based [38, 39, 19, 17, 10] and grouping-based [35, 1, 36, 26, 2, 33].”
Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers (2021, methodology): “proaches [28, 1, 41, 44, 3, 29] regard segmentation as a bottom-up grouping task by first producing pixel-wise predictions followed by grouping object instances in the postprocessing stage.”