PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud
By Shaoshuai Shi, Xiaogang Wang, Hongsheng Li. 2018
2,940 citations recorded by the source corpus.
In this paper, we propose PointRCNN for 3D object detection from raw point cloud. The whole framework is composed of two stages: stage-1 for the bottom-up 3D proposal generation and stage-2 for refining proposals in the canonical coordinates to obtain the final detection results. Instead of generating proposals from RGB image or projecting point cloud to bird's view or voxels as previous methods do, our stage-1 sub-network directly generates a small number of high-quality 3D proposals from point cloud in a bottom-up manner via segmenting the point cloud of the whole scene into foreground points and background. The stage-2 sub-network transforms the pooled points of each proposal to canonical coordinates to learn better local spatial features, which is combined with global semantic features of each point learned in stage-1 for accurate box refinement and confidence prediction. Extensive experiments on the 3D detection benchmark of KITTI dataset show that our proposed architecture outperforms state-of-the-art methods with remarkable margins by using only point cloud as input. The code is available at https://github.com/sshaoshuai/PointRCNN.
Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Geology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Xiaogang Wang
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Alan L. Yuille
- Jitendra Malik
- Yi Yang
In Geology
- Shaoshuai Shi
- Li Jiang
- Philipp Krahenbuhl
- Hongsheng Li
- Xingyi Zhou
- Xiaogang Wang
- Or Litany
- Jiaya Jia
Citations in the corpus
- Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection (2021, background): “…2018; Yan, Mao, and Li 2018; Yang, Luo, and Urtasun 2018; Zeng et al. 2018; Zhou and Tuzel 2018; Chen et al. 2019; Lang et al. 2019; Qi et al. 2019; Shi, Wang, and Li 2019; Yang et al. 2019; Zhou et al. 2019; Yang et al. 2020; Zheng et al. 2021) are proposed and have shown impressive performance…”
- A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, methodology): “ion (AP) in % on the KITTI 3D object detection test set car class, ordered based on moderate category accuracy. These algorithms only use pointcloud data. Algorithm T [s] Easy Moderate Hard PointRCNN [179] 0.10 85.9 75.8 68.3 PointPillars [180] 0.02 79.1 75.0 68.3 SECOND [140] 0.04 83.1 73.7 66.2 IP”
- Structure Aware Single-Stage 3D Object Detection From Point Cloud (2020, methodology): “[19] proposed to generate RoIs from purely point clouds by using a PointNet to segment the foreground points from the scene.”
- Voxel Transformer for 3D Object Detection (2021, background): “Point-based approaches [26, 19, 34, 35] directly operate and generate 3D bounding boxes on point clouds.”
- 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics (2019, methodology): “In Table IV (a), we switch the 3D detection module from [6] to [33].”
- PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation (2020, background): “In contrast, 3D point clouds are unordered and scattered in 3D space, causing extra difficulty in point cloud scene understanding [37, 41].”
- M3D-RPN: Monocular 3D Region Proposal Network for Object Detection (2019, methodology): “LiDAR 3D Detection: The use of LiDAR data has proven to be essential input for SOTA frameworks [6, 9, 10, 21, 27, 31, 37] for 3D object detection applied to urban scenes.”
- FVNet: 3D Front-View Proposal Generation for Real-Time Object Detection from Point Clouds (2019, background): “3D object detection [3, 4, 10, 32, 24, 13] has become increasingly important in many fields, such as autonomous driving [1], mobile robots [27] and virtual/augmented reality [17].”
- Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal (2019, methodology): “For multi-stage methods [13], [22], [23], subsequent works of IPOD [22] and PointRCNN [23] explore different proposal methods.”
- BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, background): “…lie in two categories based on their feature modality: i) point-based methods that directly operate on the raw LiDAR point clouds [38, 37, 36, 42, 57, 22]; and ii) transforming the original point clouds into a Euclidean feature space, such as 3D voxels [65] and feature pillar [20, 52, 59].”
- IoU Loss for 2D/3D Object Detection (2019, methodology): “Similar with the 2D detection, some one- or two-stage based 3D object detection frameworks have been developed, such as Frustum-Pointnet [8], Voxel-net [9], SECOND [10], PointPillars [11] and Point R-CNN [12].”
- RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (2020, background): “Therefor, most of the recent 3D object detection employ it in different representation to obtain the state-of-the-art model [48, 3, 9, 36, 21].”
- Physically Realizable Adversarial Examples for LiDAR Object Detection (2020, background): “Modern autonomous driving systems use deep neural networks (DNNs) to process LiDAR point clouds in order to perceive the world [20, 34, 45].”
- Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review (2020, methodology): “To demonstrate this flexibility, the fused point cloud is fed into multiple existing point cloud detectors, which are based on the PointRCNN [80], the VoxelNet [14] and the PointPillar [81].”
- Neural Scene Graphs for Dynamic Scenes (2020, methodology): “Specifically, we compare renderings from scene graphs learned from KITTI [11] annotations and the following two tracking methods: we combine PointRCNN [30], a lidar point cloud object detection method, with the 3D tracker AB3DMOT [41], and we evaluate on labels from a camera-only tracker CenterTrack”
- V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer (2022, background): “PointRCNN [32] proposes a two-stage strategy based on raw point clouds, which learns rough estimation in the first stage and then refines it with semantic attributes.”
- PointPainting: Sequential Fusion for 3D Object Detection (2019, methodology): “We implement PointPainting with three state of the art lidar-only methods that have public code: PointPillars [11], VoxelNet (SECOND) [34, 29], and PointRCNN [21].”
- From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network (2019, methodology): “…designed 3D sparse convolution based-backbone results in better 3D box recall than the PointNet++ based backbone in our preliminary PointRCNN framework [32] (as shown by experimental results in Table 1), which demonstrates the effectiveness of this new backbone for the point-wise feature learning.”