Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection

By Jiajun Deng, Shaoshuai Shi, Pei-Cian Li, Wen-gang Zhou, Yanyong Zhang, Houqiang Li. AAAI Conference on Artificial Intelligence, 2020

1,176 citations recorded by the source corpus.

Recent advances on 3D object detection heavily rely on how the 3D data are represented, i.e., voxel-based or point-based representation. Many existing high performance 3D detectors are point-based because this structure can better retain precise point positions. Nevertheless, point-level features lead to high computation overheads due to unordered storage. In contrast, the voxel-based structure is better suited for feature extraction but often yields lower accuracy because the input data are divided into grids. In this paper, we take a slightly different viewpoint --- we find that precise positioning of raw points is not essential for high performance 3D object detection and that the coarse voxel granularity can also offer sufficient detection accuracy. Bearing this view in mind, we devise a simple but effective voxel-based framework, named Voxel R-CNN. By taking full advantage of voxel features in a two-stage approach, our method achieves comparable detection accuracy with state-of-the-art point-based models, but at a fraction of the computation cost. Voxel R-CNN consists of a 3D backbone network, a 2D bird-eye-view (BEV) Region Proposal Network, and a detect head. A voxel RoI pooling is devised to extract RoI features directly from voxel features for further refinement. Extensive experiments are conducted on the widely used KITTI Dataset and the more recent Waymo Open Dataset. Our results show that compared to existing voxel-based methods, Voxel R-CNN delivers a higher detection accuracy while maintaining a real-time frame processing rate, i.e., at a speed of 25 FPS on an NVIDIA RTX 2080 Ti GPU. The code is available at https://github.com/djiajunustc/Voxel-R-CNN.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Geology

View Paper: https://www.semanticscholar.org/paper/efc669f75ad0ed2033e7c499ae2d6c8dab7489ce

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Geology

Citations in the corpus

3D Object Detection for Autonomous Driving: A Survey (2021, methodology): “+ TANet [76], AAAI′20 + VeloFCN [48], RSS′16 + SPG [77], ICCV′21 + PIXOR [49], CVPR′18 + CenterPoint [23], CVPR′21 + VoxelNet [24], CVPR′18 + BtcDet [10], AAAI′22 + SECOND [25], Sensors′18 + PointPillars [6], CVPR′19 + CIA-SSD [26], AAAI′21 + SE-SSD [27], CVPR′21 + Voxel R-CNN [28], AAAI′21 + CT3D [”
Focal Sparse Convolutional Networks for 3D Object Detection (2022, methodology): “The backbone networks in 3D object detectors [11, 36, 53] typically consist of one stem layer and 4 stages.”
Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds (2022, neutral): “The existing approaches can be roughly divided into single-stage [7, 11, 54, 55, 57, 58] and two-stage detectors [4, 36–39 , 53].”
Voxel Transformer for 3D Object Detection (2021, methodology): “Compared with the conventional methods [24, 5], our proposed Fast Voxel Query is efficient both in time and in space, and our approach remarkably accelerates the computation of voxel self-attention.”
Improving 3D Object Detection with Channel-wise Transformer (2021, methodology): “PV-RCNN [24] and its variant VoxelRCNN [4] adopted 3D voxel CNN as RPN to generate highquality proposals and then utilize PointNet to aggregate the voxel features around the grids.”
3D Object Detection for Autonomous Driving: A Comprehensive Survey (2022, methodology): “A similar voxel encoding strategy has been adopted by a series of following works [385, 81, 350, 297, 375, 129, 57, 254].”
VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking (2023, methodology): “Other state-of-the-art methods, including PV-Table RCNN [41], Voxel R-CNN [12], and VoTr [35], still keep the sparse-to-dense scheme to enlarge the receptive ﬁeld.”
Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, background): “Final predictions are usually generated on top of the bird-eye view (BEV) space with the flatted height axis [22, 23, 24].”