Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds

By Bo Yang, Jianan Wang, R. Clark, Qingyong Hu, Sen Wang, Andrew Markham, Agathoniki Trigoni. Neural Information Processing Systems, 2019

390 citations recorded by the source corpus.

We propose a novel, conceptually simple and general framework for instance segmentation on 3D point clouds. Our method, called 3D-BoNet, follows the simple design philosophy of per-point multilayer perceptrons (MLPs). The framework directly regresses 3D bounding boxes for all instances in a point cloud, while simultaneously predicting a point-level mask for each instance. It consists of a backbone network followed by two parallel network branches for 1) bounding box regression and 2) point mask prediction. 3D-BoNet is single-stage, anchor-free and end-to-end trainable. Moreover, it is remarkably computationally efficient as, unlike existing approaches, it does not require any post-processing steps such as non-maximum suppression, feature sampling, clustering or voting. Extensive experiments show that our approach surpasses existing work on both ScanNet and S3DIS datasets while being approximately 10x more computationally efficient. Comprehensive ablation studies demonstrate the effectiveness of our design.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Geology

View Paper: https://www.semanticscholar.org/paper/c40be1a8d5c1d759e21b73bd77506da855ea92e7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Geology

Citations in the corpus

Sparse R-CNN: End-to-End Object Detection with Learnable Proposals (2020, methodology): “Sparse R-CNN applies set prediction loss [3, 33, 44] on the fixed-size set of predictions of classification and box coordinates.”
SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration (2020, background): “Accurate matching of partial 3D surfaces is critical for point cloud registration [17, 38, 6, 21, 29], segmentation [58, 28, 27], and recognition [24, 12, 47].”
Unsupervised Point Cloud Pre-training via Occlusion Completion (2020, background): “There has been a ﬂurry of exciting new point cloud models for object detection [27, 52, 64] and segmentation [22, 26, 57, 65].”
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution (2020, background): “ns on either geometric [22,28,36,45,48,49,59,61] or semantic [56] neighborhood. There are also 3D models tailored for specic tasks such as detection [31,32,34,40{42,62, 64] and instance segmentation [13,17,18,63] built upon these modules. Recently, a few researchers started to pay attention to the e”
Review: deep learning on 3D point clouds (2020, background): “Some state-of-the-art works on instance segmentation on point cloud are [89, 90, 91, 92, 93] which are built on PointNet/PointNet++ feature learning backbone.”
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds (2019, background): “It would be interesting to extend our framework for the end-to-end 3D instance segmentation on large-scale point clouds by drawing on the recent work [64] and also for the real-time dynamic point cloud processing [35].”
OpenMask3D: Open-Vocabulary 3D Instance Segmentation (2023, background): “3D instance segmentation goes a step further by distinguishing multiple objects belonging to the same semantic category, predicting individual masks for each object instance [11, 14, 22, 24, 31, 37, 55, 60, 62, 67].”
Transformer-Based Visual Segmentation: A Survey (2023, methodology): “PSS is commonly achieved using the Point-Net [100], [101], while PIS can be achieved through two approaches: top-down approaches [102], [103] and bottom-up approaches[104],[105].”
PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation (2020, background): “[53] present the 3DBoNet that directly predicts 3D bounding boxes and pointlevel masks simultaneously per instance.”
Mask3D: Mask Transformer for 3D Semantic Instance Segmentation (2022, methodology): “Numerous methods have been proposed for 3D instance semantic segmentation, including bottom-up approaches [12, 28, 32, 58, 59], top-down approaches [22, 61, 61], and more recently, voting-based approaches [4, 13, 18, 26, 56].”
Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts (2020, neutral): “Methods mAP@0.5 ASIS [61] 55.3 3D-BoNet [69] 57.5 PointGroup [34] 57.8 3D-MPA [15] 63.”
Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review (2020, neutral): “3D-BoNet [Yang et al., 2019a]”
ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language (2019, methodology): “As an alternative to regular grids, point-based methods, such as PointNet [50] or PointNet++ [51], have been used as backbones for 3D detection and/or object instance segmentation [65, 14].”
OccuSeg: Occupancy-Aware 3D Instance Segmentation (2020, methodology): “Note that all the other methods are evaluated based on their released codes according to [46].”
SoftGroup for 3D Instance Segmentation on Point Clouds (2022, methodology): “[46] propose 3D-BoNet which directly outputs a set of bounding boxes without anchor generation and non-maximum suppression, then segments the object by a pointwise binary classifier.”