PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection

By Shaoshuai Shi, Li Jiang, Jiajun Deng, Zhe Wang, Chaoxu Guo, Jianping Shi, Xiaogang Wang, Hongsheng Li. International Journal of Computer Vision, 2021

601 citations recorded by the source corpus.

3D object detection is receiving increasing attention from both industry and academia thanks to its wide applications in various fields. In this paper, we propose Point-Voxel Region-based Convolution Neural Networks (PV-RCNNs) for 3D object detection on point clouds. First, we propose a novel 3D detector, PV-RCNN, which boosts the 3D detection performance by deeply integrating the feature learning of both point-based set abstraction and voxel-based sparse convolution through two novel steps, i.e. , the voxel-to-keypoint scene encoding and the keypoint-to-grid RoI feature abstraction. Second, we propose an advanced framework, PV-RCNN++, for more efficient and accurate 3D object detection. It consists of two major improvements: sectorized proposal-centric sampling for efficiently producing more representative keypoints, and VectorPool aggregation for better aggregating local point features with much less resource consumption. With these two strategies, our PV-RCNN++ is about $$3\times $$ 3 × faster than PV-RCNN, while also achieving better performance. The experiments demonstrate that our proposed PV-RCNN++ framework achieves state-of-the-art 3D detection performance on the large-scale and highly-competitive Waymo Open Dataset with 10 FPS inference speed on the detection range of $$150m \times 150m$$ 150 m × 150 m .

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3c3fe084caa8fd3798c1ae1a48909b9eb2d4dc5f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Song Han
2. Zhijian Liu
3. Hang Zhao
4. Zhaoxiang Zhang
5. Daniela Rus
6. Haotian Tang
7. Alexander Amini
8. Lue Fan

In Computer Vision and Pattern Recognition

1. Song Han
2. Luc Van Gool
3. Dacheng Tao
4. Xiaogang Wang
5. Trevor Darrell
6. Ming-Hsuan Yang
7. Zhijian Liu
8. Yi Yang

Citations in the corpus

• Embracing Single Stride 3D Object Detector with Sparse Transformer (2021, background): “Some hybrid approaches investigate how to combine different types of representations [6, 24, 41, 42, 49, 54].”
• Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds (2022, unknown): “To overcome the drawbacks of both point-based meth-ods ( i.e. , irregular and sparse data access, poor memory locality [23]) and voxel-based methods ( i.e. , quantization loss), several methods [3, 16, 36, 37, 53] have started to learning from 3D point clouds using point-voxel joint representations.”
• BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation (2022, background): “Another stream of research [44, 9, 45, 42, 43, 21] focus on two-stage object detector design, which adds an RCNN network to existing one-stage object detectors.”