SGN: Sequential Grouping Networks for Instance Segmentation

By Shu Liu, Jiaya Jia, Sanja Fidler, Raquel Urtasun. 2017

276 citations recorded by the source corpus.

In this paper, we propose Sequential Grouping Networks (SGN) to tackle the problem of object instance segmentation. SGNs employ a sequence of neural networks, each solving a sub-grouping problem of increasing semantic complexity in order to gradually compose objects out of pixels. In particular, the first network aims to group pixels along each image row and column by predicting horizontal and vertical object breakpoints. These breakpoints are then used to create line segments. By exploiting two-directional information, the second network groups horizontal and vertical lines into connected components. Finally, the third network groups the connected components into object instances. Our experiments show that our SGN significantly outperforms state-of-the-art approaches in both, the Cityscapes dataset as well as PASCAL VOC.

Subfields: Computer Vision and Pattern Recognition, Ocean Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Ocean Engineering

Citations in the corpus

Panoptic Feature Pyramid Networks (2019, methodology): “, [31, 1, 38] use a separate network to predict instance edges, bounding boxes, and object breakpoints, respectively).”
Hybrid Task Cascade for Instance Segmentation (2019, methodology): “Other approaches include bridging category-leval and instance-level segmentation [42], learning a boundary-aware mask representation [17], and employing a sequence of neural networks to deal with different sub-grouping problems [27].”
Path Aggregation Network for Instance Segmentation (2018, background): “They learned specially designed transformation [3, 33, 38, 59] or instance boundaries [30].”
Proposal-Free Network for Instance-Level Object Segmentation (2015, neutral): “[20] trained sequential grouping networks to progressively group instance segments.”
Incremental Few-Shot Instance Segmentation (2021, neutral): “Approaches generally fall into two categories: grouping-based [4, 11, 17, 24] and proposal-based [2, 6, 15, 20] detection methods.”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, background): “We note that there are several instance segmentation works [77, 68, 76, 2, 48, 35, 56, 20, 18, 44, 37, 30, 51, 54, 6], which could be potentially extended to bottom-up panoptic segmentation.”
PointRend: Image Segmentation As Rendering (2019, background): “Alternative, bottom-up approaches group pixels to form object masks [28, 1, 22].”
MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features (2017, background): “[49] propose to sequentially solve the grouping problem and gradually compose object instances.”
Attention-Guided Unified Network for Panoptic Segmentation (2018, neutral): “2 SGN [30] VGG16 - - - 29.”
SSAP: Single-Shot Instance Segmentation With Affinity Pyramid (2019, background): “SGN [37] sequentially groups instances with three sub-networks.”
DeeperLab: Single-Shot Image Parser (2019, methodology): “We note that Covering has been used in several instance segmentation works [92, 74, 76, 93, 78].”
Affinity Derivation and Graph Merge for Instance Segmentation (2018, methodology): “SGN [37] tends to propose a sequential framework to group the instances gradually from points to lines and finally to instances, which currently achieves the best performance of proposal-free methods.”
Weakly- and Semi-Supervised Panoptic Segmentation (2018, methodology): “This is in contrast to other instance segmentation methods such as [16, 27–30].”
Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth (2019, neutral): “th. Method AP AP 50 FPS Deep Contours [27] 2.3 3.6 5 Box2Pix [26] 13.1 27.2 10.9 BAIS [8] 17.4 36.7 <1 Discriminate Loss [6] 17.5 35.9 5 DWT [3] 19.4 35.3 <3 Dynamic Net [2] 20.0 38.3 <3 SGN [16] 25.0 44.9 0.6 Mask-RCNN [9] (ﬁne) 26.2 49.9 2.2 PANet [17] 31.8 57.1 <1 ours 27.6 50.9 11 Ta”
MSeg: A Composite Dataset for Multi-Domain Semantic Segmentation (2020, background): “It segments an image into semantically meaningful instances [44], [45], [46], [47], [48].”
Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation (2019, background): “Compared to instance segmentation in images [11, 16, 44, 1, 2, 22, 26], the problem considered in this paper requires not only to segment object instances in individual frames, but also to determine instance correspondences across multiple frames.”
SAIL-VOS: Semantic Amodal Instance Level Video Object Segmentation – A Synthetic Dataset and Baselines (2019, methodology): “Another common technique is partitioning of an initial semantic segmentation [56, 5, 6, 66].”
One-Shot Instance Segmentation (2018, methodology): “ semantic segmentation [49, 55, 73, 60, 90, 9, 15, 48], where every pixel is classiﬁed into a category, instance segmentation additionally requires to discriminate between individual object instances [27, 18, 28, 62, 19, 39, 63, 72, 5, 14, 23, 29, 45, 70, 37]. Most current state-of-the-art systems a”
SOLQ: Segmenting Objects by Learning Queries (2021, methodology): “Bottom-up methods [25, 26, 27, 28] learn per-pixel embeddings via semantic label and then cluster them into instance groups.”