SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation

By Weiyue Wang, Ronald Yu, Qiangui Huang, Ulrich Neumann. 2017

616 citations recorded by the source corpus.

We introduce Similarity Group Proposal Network (SGPN), a simple and intuitive deep learning framework for 3D object instance segmentation on point clouds. SGPN uses a single network to predict point grouping proposals and a corresponding semantic class for each proposal, from which we can directly extract instance segmentation results. Important to the effectiveness of SGPN is its novel representation of 3D instance segmentation results in the form of a similarity matrix that indicates the similarity between each pair of points in embedded feature space, thus producing an accurate grouping proposal for each point. Experimental results on various 3D scenes show the effectiveness of our method on 3D instance segmentation, and we also evaluate the capability of SGPN to improve 3D object detection and semantic segmentation results. We also demonstrate its flexibility by seamlessly incorporating 2D CNN features into the framework to boost performance.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Environmental Engineering, Geology, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Environmental Engineering

In Geology

In 3D Vision & Pose

Citations in the corpus

3D Instance Segmentation of MVS Buildings (2021, background): “However, the size of its similarity matrix tends to explode as the number of points increases [8].”
PointCNN: Convolution On X-Transformed Points (2018, methodology): “ShapeNet Parts S3DIS ScanNet pIoU mpIoU mIoU OA SyncSpecCNN [55] 84.74 82.0 - - Pd-Network [22] 85.49 82.7 - - SSCN [12] 85.98 83.3 - - SPLATNet [43] 85.4 83.7 - - SpiderCNN [53] 85.3 81.7 - - SO-Net [27] 84.9 81.0 - - PCNN [3] 85.1 81.8 - - KCNet [42] 83.7 82.2 - - SpecGCN [46] 85.4 - - - Kd-Net [2”
Point-Voxel CNN for Efficient 3D Deep Learning (2019, methodology): “For example, SegCloud [40], SGPN [43], SPGraph [20], ParamConv [42], SSCN [7] and RSNet [14] are specialized in 3D semantic/instance segmentation.”
Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scene (2020, methodology): “ularity levels, we propose our modification of point-based models fitting for multitask learning. The PointNet++ is applied as backbone since its general structure and high compatibility to multitask [35, 36]. 3 CAMPUS3D DATASET We note that the Campus3D is online accessible. Not only data can be do”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, background): “There is a variety of other LiDAR point cloud processing methods not yet explored in the end-to-end context [91], [92], [98]–[101].”
Depth-aware CNN for RGB-D Segmentation (2018, background): “Recently, deep learning frameworks [4,24,25,26,27] on point cloud are introduced to address the limitations of 3D volume.”
GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud (2018, methodology): “To show the effectiveness of our approach, we follow the setting of [42] and conduct instance segmentation for 19 object categories on the colored partial point cloud.”
Flex-Convolution - Million-Scale Point-Cloud Learning Beyond Grid-Worlds (2018, methodology): “Figure 1 further reveals an exponential increase of runtime for the PointNet family [17,19,28], ours provides significant faster inference and shows better utilization for larger point clouds with a linear increase of runtime.”
PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation (2020, methodology): “Method AP50 mPrec50 mRec50 SGPN† [49] - 0.”
A Survey on Trajectory-Prediction Methods for Autonomous Driving (2022, neutral): “Note thatthehistoricaltrajectoryinformationusedinpredictionmeth-odscanbeobtainedfromtheperceptionsystem[13]andvehicle toeverythingV2X[14],andvision-basedmethodsarenotthe focusofthisreview.”
Deep Learning for 3D Point Clouds: A Survey (2019, methodology): “esentation and local geometric features of point clouds. The learned features are then combined to achieve semantic and instance segmentation. Note that, rather than heuristic GroupMerging algorithms [197], a more ﬂexible Meanshift [205] algorithm is used to group these points into instances. Altern”
PU-Net: Point Cloud Upsampling Network (2018, background): “Subsequently, many other networks were proposed for high-level analysis problems with point clouds [18, 13, 21, 34, 28].”
From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network (2019, background): “[48] calculated the similarities between points for grouping foreground points of each instance.”
SoftGroup for 3D Instance Segmentation on Point Clouds (2022, background): “Method AP AP50 mCov mWCov mPrec50 mRec50 SGPN† [41] - - 32.”
MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation (2019, background): “SGPN [20] trains a network to predict semantic labels, a similarity matrix between all pairs of points, and point-wise conﬁdence for being a seed point, from which instance segmentation results are generated.”
Atlas: End-to-End 3D Scene Reconstruction from Posed Images (2020, background): “ded by a depth sensor. A notable exception is Kimera [37], which uses multiview stereo [19] to predict depth, however, they only show results on synthetic data and ground truth 2D segmentations. SGPN [46] formulates instance segmentation as a 3D point cloud clustering problem. Predicting a similarit”
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review (2020, background): “Thus, in [12, 105, 108, 109], a simplified PointNet is”