VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
By Yin Zhou, Oncel Tuzel. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017
4,599 citations recorded by the source corpus.
Accurate detection of objects in 3D point clouds is a central problem in many applications, such as autonomous navigation, housekeeping robots, and augmented/virtual reality. To interface a highly sparse LiDAR point cloud with a region proposal network (RPN), most existing efforts have focused on hand-crafted feature representations, for example, a bird's eye view projection. In this work, we remove the need of manual feature engineering for 3D point clouds and propose VoxelNet, a generic 3D detection network that unifies feature extraction and bounding box prediction into a single stage, end-to-end trainable deep network. Specifically, VoxelNet divides a point cloud into equally spaced 3D voxels and transforms a group of points within each voxel into a unified feature representation through the newly introduced voxel feature encoding (VFE) layer. In this way, the point cloud is encoded as a descriptive volumetric representation, which is then connected to a RPN to generate detections. Experiments on the KITTI car detection benchmark show that VoxelNet outperforms the state-of-the-art LiDAR based 3D detection methods by a large margin. Furthermore, our network learns an effective discriminative representation of objects with various geometries, leading to encouraging results in 3D detection of pedestrians and cyclists, based on only LiDAR.
Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Environmental Engineering, 3D Vision & Pose
View Paper: https://arxiv.org/abs/1711.06396
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Hongsheng Li
- Shaoshuai Shi
- Xiaogang Wang
- Philipp Krahenbuhl
- Jiaya Jia
- Li Jiang
- Xingyi Zhou
- Jianping Shi
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Xiaogang Wang
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Alan L. Yuille
- Jitendra Malik
- Yi Yang
In Environmental Engineering
In 3D Vision & Pose
- Raquel Urtasun
- Leonidas J. Guibas
- Jiaya Jia
- Charles Ruizhongtai Qi
- Shaoshuai Shi
- Hongsheng Li
- Xiaogang Wang
- Binh Yang
Citations in the corpus
- Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection (2021, background): “…approaches (Li, Zhang, and Xia 2016; Chen et al. 2017; Beltrán et al. 2018; Yan, Mao, and Li 2018; Yang, Luo, and Urtasun 2018; Zeng et al. 2018; Zhou and Tuzel 2018; Chen et al. 2019; Lang et al. 2019; Qi et al. 2019; Shi, Wang, and Li 2019; Yang et al. 2019; Zhou et al. 2019; Yang et al.…”
- Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection (2018, neutral): “[6] introduce a voxel feature encoding (VFE) layer that can learn the unified features directly from the Lidar point clouds.”
- A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, methodology): “The first convincing results for point cloud-only 3D bounding box estimation were produced by VoxelNet [164].”
- RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous Driving (2019, methodology): “Other approaches divided the space into voxels [24] with a predefined resolution, projected the point cloud inside these voxels, and performed voxel-level classification.”
- TANet: Robust 3D Object Detection from Point Clouds with Triple Attention (2019, methodology): “The coarse step provides a rough estimation of the object following (Zhou and Tuzel 2018; Lang et al. 2019).”
- PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud (2018, methodology): “There are several other alternative point-cloud network structures, such as [26, 13] or VoxelNet [43] with sparse convolutions [9], which could also be adopted as our backbone network.”
- PF-Net: Point Fractal Network for 3D Point Cloud Completion (2020, background): “However, multi-view and voxel-based representation[18, 7, 34, 11, 13, 37] leads to unnecessarily voluminous and limits the output resolution of voxels.”
- Accurate Monocular 3D Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving (2019, background): “VoxelNet [42] divides point clouds into equally spaced 3D voxels and transforms a group of points within each voxel into a unified feature representation.”
- BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection (2022, methodology): “VoxelNet (Zhou and Tuzel 2018) voxelizes the point cloud, converting it from sparse to dense voxels, and then proposes bounding boxes in dense space to aid the index during convolution.”
- A Survey of End-to-End Driving: Architectures and Training Methods (2020, background): “There is a variety of other LiDAR point cloud processing methods not yet explored in the end-to-end context [91], [92], [98]–[101].”
- Embracing Single Stride 3D Object Detector with Sparse Transformer (2021, methodology): “SST voxelizes the point clouds and extracts voxel features following prior work [19, 57, 65].”
- CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection (2020, methodology): “The majority of LiDAR-based methods either use 3D voxels [12, 35] or 2D projections [13, 5, 29, 31] for point cloud representation.”
- CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection (2020, methodology): “[5] uses voxels to encode the raw point cloud, and 3D CNNs (Convolutonal Neural Networks) are applied to learn voxel features for classification and bounding box regression.”
- Generating 3D Adversarial Point Clouds (2018, methodology): “We choose PointNet because the model and its variants have been widely and successfully adopted in many applications such as 3D object detection for autonomous driving [18, 34, 30], semantic segmentation for indoor scene understanding [12, 19], and AI-assisted shape design [24].”
- ImVoteNet: Boosting 3D Object Detection in Point Clouds With Image Votes (2020, methodology): “Recently, with advances in deep networks for point cloud data, several works [33, 56, 41] have shown state-of-the-art 3D detection results with point cloud as the only input.”
- Learning Depth-Guided Convolutions for Monocular 3D Object Detection (2019, background): “For example, [59] divides point clouds into equally spaced 3D voxels and transforms a group of points within each voxel into a unified feature representation.”
- SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation (2020, methodology): “Currently, the most successful 3D object detection methods heavily depend on LiDAR point cloud [43, 30, 40] SMOKE”
- Voxel Transformer for 3D Object Detection (2021, methodology): “VoxelNet [43] utilizes a 3D CNN to extract voxel features from a dense grid.”
- MVX-Net: Multimodal VoxelNet for 3D Object Detection (2019, methodology): “We use the network as described in [43].”
- Group-Free 3D Object Detection via Transformers (2021, methodology): “Voxel-based meth-ods [37, 53] convert points into voxels and employ 3D Con-vNets to generate features for 3D box generation.”