From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network

By Shaoshuai Shi, Zhe Wang, Jianping Shi, Xiaogang Wang, Hongsheng Li. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019

1,024 citations recorded by the source corpus.

3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications. In this paper, we extend our preliminary work PointRCNN to a novel and strong point-cloud-based 3D object detection framework, the part-aware and aggregation neural network (Part-<inline-formula><tex-math notation="LaTeX">$A^2$</tex-math><alternatives><mml:math><mml:msup><mml:mi>A</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:math><inline-graphic xlink:href="shi-ieq1-2977026.gif"/></alternatives></inline-formula> net). The whole framework consists of the part-aware stage and the part-aggregation stage. First, the part-aware stage for the first time fully utilizes free-of-charge part supervisions derived from 3D ground-truth boxes to simultaneously predict high quality 3D proposals and accurate intra-object part locations. The predicted intra-object part locations within the same proposal are grouped by our new-designed RoI-aware point cloud pooling module, which results in an effective representation to encode the geometry-specific features of each 3D proposal. Then the part-aggregation stage learns to re-score the box and refine the box location by exploring the spatial relationship of the pooled intra-object part locations. Extensive experiments are conducted to demonstrate the performance improvements from each component of our proposed framework. Our Part-<inline-formula><tex-math notation="LaTeX">$A^2$</tex-math><alternatives><mml:math><mml:msup><mml:mi>A</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:math><inline-graphic xlink:href="shi-ieq2-2977026.gif"/></alternatives></inline-formula> net outperforms all existing 3D detection methods and achieves new state-of-the-art on KITTI 3D object detection dataset by utilizing only the LiDAR point cloud data.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology

View Paper: https://www.semanticscholar.org/paper/49e32810319a03f60cae281557eba2f0d91512f2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection (2019, methodology): “19 - - - - - - - - - - - Part-A(2) [28] LiDAR only 87.”
CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloud (2020, methodology): “Based on PointRCNN, Part-A 2 (Shi et al. 2020b) replaces the PointNet++ backbone with a sparse convolutional network, and proposes RoI-aware point cloud feature pooling in the reﬁnement.”
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, methodology): “Part-A2-Net (Shi et al., 2020b) aggregates the point-wise part locations by the voxel-based RoI-aware pooling to improve 3D detection performance.”
CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection (2020, background): “Besides, Part-A(2) in [23] explores predicting intra-object part locations (lower left, upper right, etc.”
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers (2022, background): “LiDAR-only 3D Detection aims to predict 3D bounding boxes of objects in given point clouds [3, 4, 27, 28, 38, 47, 54, 67, 69, 70].”
BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation (2022, background): “Another stream of research [24], [25], [26], [27], [28], [29] focuses on twostage object detector design, which adds an RCNN network to existing one-stage object detectors.”
PointMamba: A Simple State Space Model for Point Cloud Analysis (2024, background): “Point cloud analysis is one of the fundamental tasks in computer vision and has a wide range of real-world applications [47, 54, 8], including robotics, autonomous driving, and augmented reality.”
Learning Distilled Collaboration Graph for Multi-Agent Perception (2021, background): “Comprehensive experiments conducted in 3D object detection [37, 28, 29, 17] have shown that the proposed DiscoNet achieves better performance-bandwidth trade-off and lower communication latency than the state-of-the-art intermediate collaboration methods.”