GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud

By Li Yi, Wang Zhao, He Wang, Minhyuk Sung, Leonidas J. Guibas. 2018

370 citations recorded by the source corpus.

We introduce a novel 3D object proposal approach named Generative Shape Proposal Network (GSPN) for instance segmentation in point cloud data. Instead of treating object proposal as a direct bounding box regression problem, we take an analysis-by-synthesis strategy and generate proposals by reconstructing shapes from noisy observations in a scene. We incorporate GSPN into a novel 3D instance segmentation framework named Region-based PointNet (R-PointNet) which allows flexible proposal refinement and instance segmentation generation. We achieve state-of-the-art performance on several 3D instance segmentation tasks. The success of GSPN largely comes from its emphasis on geometric understandings during object proposal, greatly reducing proposals with low objectness.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

3D Instance Segmentation of MVS Buildings (2021, neutral): “GSPN extends the structure of Mask R-CNN (that was originally developed for images) to process 3D data [7].”
ImVoteNet: Boosting 3D Object Detection in Point Clouds With Image Votes (2020, background): “In the context of 3D scene understanding, there have also been promising results on semantic and instance segmentation [53, 4, 9].”
UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders (2020, methodology): “Recently, VAE algorithms have been extened to 3D domain targeting applications such as 3D meshes deformation [52], and point cloud instance segmentation [59].”
MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation (2019, background): “GSPN [22] presents a generative model for generating proposals.”
JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds (2019, methodology): “GSPN (Yi et al. 2019) generates proposals by reconstructing shapes and outputs the ﬁnal segmentation results based on PointNet++. 3D-BoNet (Yang et al. 2019) directly regresses 3D bounding boxes and predicts point-level masks for all instances simultaneously.”
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding (2020, background): “their corresponding object labels (recognition). This calls for an architectural modication as the SR-UNet architecture does not directly output bounding box coordinates. Among many dierent choices [79, 29, 46, 44], we identify the recently proposed VoteNet [45] as a good candidate for three main re”
Learning 3D Semantic Scene Graphs From 3D Indoor Reconstructions (2020, background): “Current 3D scene understanding works include perception tasks such as instance segmentation [12, 21, 44, 50], semantic segmentation [34, 36, 5, 38] as well as 3D object detection and classification [40, 34, 35, 54].”
PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation (2020, background): “[54] propose GSPN, which takes an analysis-by-synthesis strategy to generate proposals for instance segmentation.”
OccuSeg: Occupancy-Aware 3D Instance Segmentation (2020, methodology): “Both the proposal-based methods (3DSIS [17], GSPN [48] and 3D-BoNet [46]) and the proposal-free methods (SGPN [41] ASIS [42]) are concerned.”
From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network (2019, methodology): “[46] proposed an analysis-by-synthesis strategy to generate 3D proposals for 3D instance segmentation.”
Deep Hough Voting for 3D Object Detection in Point Clouds (2019, methodology): “GSPN [54] is a recent instance segmentation method using a generative model to propose object instances, which is also based on a PointNet++ backbone.”
Transformer-Based Visual Segmentation: A Survey (2023, methodology): “PSS is commonly achieved using the Point-Net [100], [101], while PIS can be achieved through two approaches: top-down approaches [102], [103] and bottom-up approaches[104],[105].”
An End-to-End Transformer Model for 3D Object Detection (2021, methodology): “We focus on methods that directly use 3D point clouds [40, 51, 75, 85].”
Self-Supervised Pretraining of 3D Features on any Point-Cloud (2021, methodology): “Point-cloud based models [66, 68] are widely used in classification and segmentation tasks [6, 40, 45, 66, 68, 91, 98, 100, 106, 107], 3D reconstruction [23, 90, 110] and 3D object detection [63, 64, 77, 98, 112, 112, 114].”
SoftGroup for 3D Instance Segmentation on Point Clouds (2022, background): “Backbone (GPU): 1612 GSPN [47] Point sampling (GPU): 9559 12702 Neighbour search (CPU): 1500”
Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts (2020, neutral): “A variety of datasets [2, 13, 54, 18, 56] and algorithms have been proposed for 3D object detection [45, 44, 43, 24], semantic segmentation [46, 57, 63, 58, 20, 12] and instance segmentation [60, 30, 71, 36, 61, 69, 31, 15, 34, 33].”
On the Limitations of Representing Functions on Sets (2019, background): “Much of the recent work on deep learning with unordered sets follows the paradigm discussed in (Ravanbakhsh et al., 2016 and Yi et al. (2018) show that set-based models can also be used to learn generative models of point clouds.”
Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds (2019, methodology): “This simple box fusing approach is extremely computationally efficient, compared with the commonly used RoIAlign in prior art [58; 15; 13] which involves the expensive point feature sampling and alignment.”
ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data (2021, background): “A variety of methods are addressing different parts of the challenge, like depth estimation, 3D reconstruction, instance segmentation [1, 2, 3], object detection [4, 5, 6, 7] and more.”