Deep Hough Voting for 3D Object Detection in Point Clouds
By Charles Ruizhongtai Qi, Or Litany, Kaiming He, Leonidas J. Guibas. 2019
1,503 citations recorded by the source corpus.
Current 3D object detection methods are heavily influenced by 2D detectors. In order to leverage architectures in 2D detectors, they often convert 3D point clouds to regular grids (i.e., to voxel grids or to bird's eye view images), or rely on detection in 2D images to propose 3D boxes. Few works have attempted to directly detect objects in point clouds. In this work, we return to first principles to construct a 3D detection pipeline for point cloud data and as generic as possible. However, due to the sparse nature of the data -- samples from 2D manifolds in 3D space -- we face a major challenge when directly predicting bounding box parameters from scene points: a 3D object centroid can be far from any surface point thus hard to regress accurately in one step. To address the challenge, we propose VoteNet, an end-to-end 3D object detection network based on a synergy of deep point set networks and Hough voting. Our model achieves state-of-the-art 3D detection on two large datasets of real 3D scans, ScanNet and SUN RGB-D with a simple design, compact model size and high efficiency. Remarkably, VoteNet outperforms previous methods by using purely geometric information without relying on color images.
Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
- Andreas Geiger
- Or Litany
- Matthias Nießner
- Ishan Misra
- Rohit Girdhar
- Armand Joulin
- Jiaya Jia
- Leonidas J. Guibas
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Ming-Hsuan Yang
- Yi Yang
- Andrea Vedaldi
In Geology
- Or Litany
- Matthias Nießner
- Jiaya Jia
- Rohit Girdhar
- Li Jiang
- Armand Joulin
- Ishan Misra
- Charles Ruizhongtai Qi
Citations in the corpus
- Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection (2021, methodology): “…Beltrán et al. 2018; Yan, Mao, and Li 2018; Yang, Luo, and Urtasun 2018; Zeng et al. 2018; Zhou and Tuzel 2018; Chen et al. 2019; Lang et al. 2019; Qi et al. 2019; Shi, Wang, and Li 2019; Yang et al. 2019; Zhou et al. 2019; Yang et al. 2020; Zheng et al. 2021) are proposed and have shown…”
- FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation (2021, neutral): “These approaches either utilize 3D ConvNets [56, 57] or point cloud network [74, 47, 24] for feature extraction and 3D bounding box prediction.”
- Self-Supervised Pretraining of 3D Features on any Point-Cloud (2021, methodology): “We first adopt standard pointcloud data argumentation methods proposed in [64], which are random point up/down sampling, random flip in xy axis, and random rotation.”
- Structure Aware Single-Stage 3D Object Detection From Point Cloud (2020, background): “ods [16, 19, 2, 28, 15, 20] can leverage finer spatial information at the second stage, which focuses only on the regions of interest (RoIs) predicted by the first stage, and consequently predicts more accurate bounding box.”
- KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D (2021, methodology): “Following [76], the mean AP is calculated at a threshold of 0.”
- OccuSeg: Occupancy-Aware 3D Instance Segmentation (2020, methodology): “VoteNet [35] predicts offset vectors to the corresponding object centers for seed points, followed by a clustering module to generate object proposals.”
- Center-based 3D Object Detection and Tracking (2020, methodology): “also compare to a grid point-based representation, as used in VoteNet [36], PointRCNN [43], and PIXOR [55], which assigns all points inside the ground truth box as positive.”
- 3DSSD: Point-Based 3D Single Stage Object Detector (2020, methodology): “They are shifted under the supervision of their relative locations to their corresponding instances as illustrated in Figure 2, same as the way of VoteNet [19].”
- PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, neutral): “(Qi et al., 2019) proposes hough voting strategy for feature grouping.”
- Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds (2019, methodology): “Without relying on anchors, the recent PointRCNN [44] learns to detect via foreground point segmentation, and the VoteNet [35] detects objects via point feature grouping, sampling and voting.”
- TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers (2022, background): “LiDAR-only 3D Detection aims to predict 3D bounding boxes of objects in given point clouds [3, 4, 27, 28, 38, 46, 53, 66, 68, 69].”
- ScanQA: 3D Question Answering for Spatial Scene Understanding (2021, positive): “We pro-pose a 3D-QA model with textual and 3D scene encoding and several baseline models, including 2D image models (2D-QA), a combination of 3D object localization models [10, 38], and a question answering model [51].”
- Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution (2020, background): “There are also 3D models tailored for specific tasks such as detection [38, 39, 41, 47–49, 70, 73] and instance segmentation [16, 21, 23, 71] built upon these modules.”
- Review: deep learning on 3D point clouds (2020, methodology): “[108] used PointNet++ to learn point wise features which 11”
- Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, background): “Point-based detectors usually aggregate features from raw point clouds with set abstraction [13] and then predict box proposals [14, 15, 16].”
- PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding (2020, methodology): “For the detection experiment, we again follow VoteNet [45] and simply switch the original backbone network with the SR-UNet without other modifications to the detection head (See Appendix for details).”
- ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language (2019, methodology): “Following Qi et al. [46], we also append the height of the point from the ground and normals to the point features.”
- An End-to-End Transformer Model for 3D Object Detection (2021, methodology): “VoteNet [42] uses Hough Voting on sparse point cloud inputs and detects boxes by feature sampling, grouping and voting operations designed for 3D data.”
- Point Transformer (2020, background): “The three-dimensional information has a wide range of applications in autonomous driving [1]–[6] and computer vision [7], [8].”
- PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation (2019, background): “The extra depth information allows 2D algorithms to be extend into 3D space with better performance, like Point-Fusion [53], Frustum pointnets[39] and VoteNet[38].”