SoftGroup for 3D Instance Segmentation on Point Clouds

By Thang Vu, Kookhoi Kim, Tung Minh Luu, Xuan Thanh Nguyen, Chang-Dong Yoo. Computer Vision and Pattern Recognition, 2022

350 citations recorded by the source corpus.

Existing state-of-the-art 3D instance segmentation methods perform semantic segmentation followed by grouping. The hard predictions are made when performing semantic segmentation such that each point is associated with a single class. However, the errors stemming from hard decision propagate into grouping that results in (1) low overlaps between the predicted instance with the ground truth and (2) substantial false positives. To address the aforementioned problems, this paper proposes a 3D instance segmentation method referred to as SoftGroup by performing bottom-up soft grouping followed by top-down refinement. SoftGroup allows each point to be associated with multiple classes to mitigate the problems stemming from semantic prediction errors and suppresses false positive instances by learning to categorize them as background. Experimental results on different datasets and multiple evaluation metrics demonstrate the efficacy of SoftGroup. Its performance surpasses the strongest prior method by a significant margin of $+6.2\%$ on the ScanNet v2 hidden test set and $+6.8\%$ on S3DIS Area 5 in terms of $AP_{50}$. Soft-Group is also fast, running at 345ms per scan with a sin-gle Titan X on ScanNet v2 dataset. The source code and trained models for both datasets are available at https://github.com/thangvubk/SoftGroup.git.

Subfields: Aerospace Engineering, Computational Mechanics, Geology

View Paper: https://www.semanticscholar.org/paper/a35420f502f1104db871c946db1e2ddf44dfe58c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computational Mechanics

In Geology

Citations in the corpus

ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding (2022, background): “3D visual understanding research [6, 12, 16, 17, 24, 25] is drawing significant attention in recent years due to the increasing demand of real-world applications such as augmented/virtual reality [1, 27, 31, 46], autonomous driving [23, 57] and robotics [2, 48].”
Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following (2023, background): “In these years, 3D vision has gained significant attention and development, driven by the rising popularity of autonomous driving [11, 69, 71], navigation [72, 76, 98], 3D scene understanding [2, 43, 46, 74], and robotics [31, 67].”
ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes (2023, methodology): “3D Instance Segmentation We compare 3 methods for 3D instance segmentation on ScanNet++: PointGroup [16], HAIS [7] and SoftGroup [44] on 75 instances classes - semantic classes excluding wall, ceiling and floor.”
OpenMask3D: Open-Vocabulary 3D Instance Segmentation (2023, background): “3D instance segmentation goes a step further by distinguishing multiple objects belonging to the same semantic category, predicting individual masks for each object instance [11, 14, 22, 24, 31, 37, 55, 60, 62, 67].”