PointPillars: Fast Encoders for Object Detection From Point Clouds

By Alex H. Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, Oscar Beijbom. 2018

4,614 citations recorded by the source corpus.

Object detection in point clouds is an important aspect of many robotics applications such as autonomous driving. In this paper, we consider the problem of encoding a point cloud into a format appropriate for a downstream detection pipeline. Recent literature suggests two types of encoders; fixed encoders tend to be fast but sacrifice accuracy, while encoders that are learned from data are more accurate, but slower. In this work, we propose PointPillars, a novel encoder which utilizes PointNets to learn a representation of point clouds organized in vertical columns (pillars). While the encoded features can be used with any standard 2D convolutional detection architecture, we further propose a lean downstream network. Extensive experimentation shows that PointPillars outperforms previous encoders with respect to both speed and accuracy by a large margin. Despite only using lidar, our full detection pipeline significantly outperforms the state of the art, even among fusion methods, with respect to both the 3D and bird’s eye view KITTI benchmarks. This detection performance is achieved while running at 62 Hz: a 2 - 4 fold runtime improvement. A faster version of our method matches the state of the art at 105 Hz. These benchmarks suggest that PointPillars is an appropriate encoding for object detection in point clouds.

Subfields: Aerospace Engineering, Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection (2021, background): “…Chen et al. 2017; Beltrán et al. 2018; Yan, Mao, and Li 2018; Yang, Luo, and Urtasun 2018; Zeng et al. 2018; Zhou and Tuzel 2018; Chen et al. 2019; Lang et al. 2019; Qi et al. 2019; Shi, Wang, and Li 2019; Yang et al. 2019; Zhou et al. 2019; Yang et al. 2020; Zheng et al. 2021) are proposed and…”
Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving (2023, neutral): “3D object detection is often performed in LiDAR point clouds [55, 18, 51, 52, 34, 36, 10, 39].”
DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection (2022, methodology): “Firstly, we choose PointPillars [15] as the 3D detector and train the two detectors with infrastructure-view and vehicle-view data in VIC-Sync separately.”
BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View (2021, methodology): “There are some pioneers [32, 18], who have exploited the 3D object detection task in BEV.”
Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks (2021, background): “Thus, many state-of-the-art LiDAR-based AD perception model designs choose to use aggregated input features such as average height and intensity of the 3D points grouped at the level of 3D cells , or voxels [30]–[32], [49].”
PointPainting: Sequential Fusion for 3D Object Detection (2019, methodology): “We implement PointPainting with three state of the art lidar-only methods that have public code: PointPillars [11], VoxelNet (SECOND) [34, 29], and PointRCNN [21].”
Scalability in Perception for Autonomous Driving: Waymo Open Dataset (2019, methodology): “For our baseline, we use our already trained PointPillars [16] models from above, 1− IOU as our cost function, the Hungarian method [15] as our assignment function, and a Kalman Filter [13] as our state update function.”
DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models (2024, background): “Existing autonomous driving systems, typically comprising 3D perception, motion prediction, and planning, struggle with these scene understanding challenges.”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, background): “[145] argue that the hard-coded features for BEV representation may not be optimal.”
ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection (2021, background): “Recently, this field has obtained remarkable advancements [46, 24, 36, 37, 34, 35] driven by deep neural networks and large-scale human-annotated datasets [13, 38].”
PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation (2020, methodology): “PointPillars [16] improved this idea by adding a PointNet model on the BEV representation.”
DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection (2022, methodology): “DeepFusion serves as a plug-in that can be easily applied to most of the voxel-based 3D detection methods, such as PointPillars [17] and CenterPoint [44].”
CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers (2022, positive): “We select PointPillar [57] as the point cloud feature extractor and set the voxel resolution as (0.4, 0.4, 4) on the x, y, and z-axis.”
Argoverse: 3D Tracking and Forecasting With Rich Maps (2019, methodology): “[31] Alex H.”