PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation

By Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia. Computer Vision and Pattern Recognition, 2020

584 citations recorded by the source corpus.

Instance segmentation is an important task for scene understanding. Compared to the fully-developed 2D, 3D instance segmentation for point clouds have much room to improve. In this paper, we present PointGroup, a new end-to-end bottom-up architecture, specifically focused on better grouping the points by exploring the void space between objects. We design a two-branch network to extract point features and predict semantic labels and offsets, for shifting each point towards its respective instance centroid. A clustering component is followed to utilize both the original and offset-shifted point coordinate sets, taking advantage of their complementary strength. Further, we formulate the ScoreNet to evaluate the candidate instances, followed by the Non-Maximum Suppression (NMS) to remove duplicates. We conduct extensive experiments on two challenging datasets, ScanNet v2 and S3DIS, on which our method achieves the highest performance, 63.6% and 64.0%, compared to 54.9% and 54.4% achieved by former best solutions in terms of mAP with IoU threshold 0.5.

Subfields: Aerospace Engineering, Computational Mechanics, Geology

View Paper: https://www.semanticscholar.org/paper/9712cc6fc96f463842f9d41c565e3a8781bfba40

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Marc Pollefeys
2. Matthias Nießner
3. Angela Dai
4. Or Litany
5. Andreas Geiger
6. Hongsheng Li
7. Bastian Leibe
8. Dieter Fox

In Computational Mechanics

1. Andreas Geiger
2. Matthias Nießner
3. Or Litany
4. Angela Dai
5. Yiyi Liao
6. Ji Hou
7. Marc Pollefeys
8. Hongsheng Li

In Geology

1. Or Litany
2. Matthias Nießner
3. Marc Pollefeys
4. Angela Dai
5. Ji Hou
6. Francis Engelmann
7. Andreas Geiger
8. Arsalan Mousavian

Citations in the corpus

• ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes (2023, methodology): “3D Instance Segmentation We compare 3 methods for 3D instance segmentation on ScanNet++: PointGroup [16], HAIS [7] and SoftGroup [44] on 75 instances classes - semantic classes excluding wall, ceiling and floor.”
• Unseen Object Instance Segmentation for Robotic Environments (2020, methodology): “We train PointGroup [13] for 300k iterations with a batch size of 4 on TOD.”
• Transformer-Based Visual Segmentation: A Survey (2023, methodology): “PSS is commonly achieved using the Point-Net [100], [101], while PIS can be achieved through two approaches: top-down approaches [102], [103] and bottom-up approaches[104],[105].”
• Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts (2020, methodology): “We do not use any special modules such as Pro-posal Aggregation [15] or Scoring Network [34].”
• STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos (2020, methodology): “Recent methods for 3D object detection and instance segmentation also follow this trend [22,41,74,21,106].”
• KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D (2021, methodology): “The qualitative result of PointGroup is shown in Fig.”
• Point Transformer V3: Simpler, Faster, Stronger (2023, methodology): “For indoor instance segmentation, we use PointGroup [35] as our foundational framework, a popular choice in 3D representation learning [30, 91, 92, 94].”
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation (2020, methodology): “It can be found that our method achieves much better performance than existing methods [22, 15].”
• Language-Grounded Indoor 3D Semantic Segmentation in the Wild (2022, methodology): “For the task and clustering algo-rithm, we adopt the paradigms of [24,20] to our ScanNet200 benchmark.”
• OpenMask3D: Open-Vocabulary 3D Instance Segmentation (2023, background): “3D instance segmentation goes a step further by distinguishing multiple objects belonging to the same semantic category, predicting individual masks for each object instance [11, 14, 22, 24, 31, 37, 55, 60, 62, 67].”
• SoftGroup for 3D Instance Segmentation on Point Clouds (2022, methodology): “Backbone (GPU): 128 PointGroup [15] Clustering (GPU+CPU):221 452 ScoreNet (GPU): 103”
• Mask3D: Mask Transformer for 3D Semantic Instance Segmentation (2022, methodology): “, centers [26], bounding boxes [7], occupancy [18]), and heuristics for clustering the”
• Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution (2020, background): “There are also 3D models tailored for specific tasks such as detection [38, 39, 41, 47–49, 70, 73] and instance segmentation [16, 21, 23, 71] built upon these modules.”