PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
By Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, Hongsheng Li. Computer Vision and Pattern Recognition, 2019
2,288 citations recorded by the source corpus.
We present a novel and high-performance 3D object detection framework, named PointVoxel-RCNN (PV-RCNN), for accurate 3D object detection from point clouds. Our proposed method deeply integrates both 3D voxel Convolutional Neural Network (CNN) and PointNet-based set abstraction to learn more discriminative point cloud features. It takes advantages of efficient learning and high-quality proposals of the 3D voxel CNN and the flexible receptive fields of the PointNet-based networks. Specifically, the proposed framework summarizes the 3D scene with a 3D voxel CNN into a small set of keypoints via a novel voxel set abstraction module to save follow-up computations and also to encode representative scene features. Given the high-quality 3D proposals generated by the voxel CNN, the RoI-grid pooling is proposed to abstract proposal-specific features from the keypoints to the RoI-grid points via keypoint set abstraction. Compared with conventional pooling operations, the RoI-grid feature points encode much richer context information for accurately estimating object confidences and locations. Extensive experiments on both the KITTI dataset and the Waymo Open dataset show that our proposed PV-RCNN surpasses state-of-the-art 3D detection methods with remarkable margins.
Subfields: Aerospace Engineering, Computational Mechanics, Geology
View Paper: https://www.semanticscholar.org/paper/199123c3a93521a397ee4f2ae8c852b0b252f5ba
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Jian‐Yuan Sun
- Hongsheng Li
- Xingyi Zhou
- Philipp Krahenbuhl
- Tianwei Yin
- Hang Zhao
- Xiaogang Wang
- Yulan Guo
In Computational Mechanics
- Li Jiang
- Xingyi Zhou
- Philipp Krahenbuhl
- Ishan Misra
- Rohit Girdhar
- Armand Joulin
- Chi-Wing Fu
- Jiaya Jia
In Geology
- Li Jiang
- Philipp Krahenbuhl
- Xingyi Zhou
- Tianwei Yin
- Shaoshuai Shi
- Xiaogang Wang
- Chi-Wing Fu
- Hongsheng Li
Citations in the corpus
- One Million Scenes for Autonomous Driving: ONCE Dataset (2021, methodology): “Many techniques have been explored for 3D object detection in driving scenarios, and they can be broadly categorized into two classes: 1) Single-modality 3D detectors [49, 36, 24, 37, 52, 57, 35, 50] are designed to detect objects solely from sparse point clouds.”
- BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, background): “Recently, people started to exploit these two feature modalities in a single model to increase the representation power [4, 64, 41].”
- ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data (2021, background): “The majority of the published techniques focus on outdoor environments [6, 35] mainly in the context of autonomous driving, where diverse datasets such as [19, 13] exists.”
- Voxel Transformer for 3D Object Detection (2021, methodology): “Data augmentations and other configurations are kept the same as the corresponding baselines [34, 26].”
- Self-Supervised Pretraining of 3D Features on any Point-Cloud (2021, background): “There are also efforts to combine different 3D input representations [29, 79, 100, 117–119].”
- ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding (2022, methodology): “One is projecting a point cloud into 3D voxels [30, 42] and then using 2D/3D convolutions for feature extraction.”
- V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer (2022, methodology): “To satisfy both computational and flexible receptive field requirements, [30,52,47] combine voxel-based and pointbased approaches to detect 3D objects.”
- Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework (2022, background): “Since the point cloud data structure is irregular and unordered, some works consider projecting the original point clouds to intermediate voxels (Maturana & Scherer, 2015; Shi et al., 2020) or images (You et al.”
- Point2Seq: Detecting 3D Objects as Sequences (2022, methodology): “Our approach even outperforms those time-consuming two-stage 3D detectors [16,24,25], which indicates the effectiveness of the scene-to-sequence decoder.”
- ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection (2021, neutral): “To be noted, IoU threshold 0.7 is a more strict criterion and widely adopted to assess 3D object detection models for the “car” category [9, 12, 8, 11].”
- BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation (2022, background): “Another stream of research [24], [25], [26], [27], [28], [29] focuses on twostage object detector design, which adds an RCNN network to existing one-stage object detectors.”
- Multimodal Virtual Point 3D Detection (2021, methodology): “Two-stage 3D detectors [8, 45– 47 , 63] use a PointNet-based PointNet-based Camera-based 3D Object Detection Camera-based 3D object detection predicts 3D bounding boxes from camera images.”
- 3D Object Detection with Pointformer (2020, methodology): “Hybrid methods combine both types of features at the local level [41, 15] or at the network level [39, 24].”
- PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, methodology): “…2D pixels (Chen et al., 2017; Yang et al., 2018b) or 3D voxels (Zhou and Tuzel, 2018; Chen et al., 2019b) for processing them with 2D/3D CNN. Sparse convolution (Graham et al., 2018) is adopted in (Yan et al., 2018; Shi et al., 2020b) to effectively learn sparse voxel features from point clouds.”
- Embracing Single Stride 3D Object Detector with Sparse Transformer (2021, background): “Some hybrid approaches investigate how to combine different types of representations [6, 24, 41, 42, 49, 54].”
- Deep Learning for 3D Point Clouds: A Survey (2019, methodology): “[148] proposed PointVoxel-RCNN (PV-RCNN) to leverage both 3D convolutional network and PointNetbased set abstraction for the learning of point cloud fea-”
- 3D Shape Generation and Completion through Point-Voxel Diffusion (2021, positive): “We therefore explore a separate point-voxel representation [22, 33], and our work is most related to point-voxel CNN [24], which proposes to voxelize the point clouds for 3D convolution.”
- Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection (2020, methodology): “PV-RCNN (Shi et al. 2020a) extends SECOND by adding a keypoints branch to preserve 3D structural information.”