PIXOR: Real-time 3D Object Detection from Point Clouds

By Binh Yang, Wenjie Luo, Raquel Urtasun. 2018

1,233 citations recorded by the source corpus.

We address the problem of real-time 3D object detection from point clouds in the context of autonomous driving. Speed is critical as detection is a necessary component for safety. Existing approaches are, however, expensive in computation due to high dimensionality of point clouds. We utilize the 3D data more efficiently by representing the scene from the Bird's Eye View (BEV), and propose PIXOR, a proposal-free, single-stage detector that outputs oriented 3D object estimates decoded from pixel-wise neural network predictions. The input representation, network architecture, and model optimization are specially designed to balance high accuracy and real-time efficiency. We validate PIXOR on two datasets: the KITTI BEV object detection benchmark, and a large-scale 3D vehicle detection benchmark. In both datasets we show that the proposed detector surpasses other state-of-the-art methods notably in terms of Average Precision (AP), while still runs at 10 FPS.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Static-Dynamic Co-Teaching for Class-Incremental 3D Object Detection (2021, background): “Many deep learning-based approaches (Li, Zhang, and Xia 2016; Chen et al. 2017; Beltrán et al. 2018; Yan, Mao, and Li 2018; Yang, Luo, and Urtasun 2018; Zeng et al. 2018; Zhou and Tuzel 2018; Chen et al. 2019; Lang et al. 2019; Qi et al. 2019; Shi, Wang, and Li 2019; Yang et al. 2019; Zhou et al.…”
Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection (2018, background): “[10] project the Lidar point clouds into birds’ eye view image and propose a road detector and a car detector, respectively.”
MP3: A Unified Model to Map, Perceive, Predict and Plan (2021, methodology): “This network combines ideas from [9,53] to extract geometric, semantic and motion information about the scene.”
Deep Continuous Fusion for Multi-sensor 3D Object Detection (2018, methodology): “Our continuous fusion model, its BEV baseline, and PIXOR [37] are compared.”
TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers (2022, background): “LiDAR-only 3D Detection aims to predict 3D bounding boxes of objects in given point clouds [3, 4, 27, 28, 38, 47, 54, 67, 69, 70].”
Multi-Task Multi-Sensor Fusion for 3D Object Detection (2019, methodology): “We demonstrate the effectiveness of our approach on the KITTI object detection benchmark [8] as well as the more challenging TOR4D object detection benchmark [34].”
IntentNet: Learning to Predict Intention from Raw Sensor Data (2018, background): “Following [16, 4, 14], we represent point clouds in bird’s eye view (BEV) as a 3D tensor, treating height as our channel dimension.”
Fast Point R-CNN (2019, methodology): “Based on this observation, our backbone network is composed of 2D and 3D convolutions, achieving high efficiency as PIXOR [40] and even higher performance than VoxelNet [43].”
SECOND: Sparsely Embedded Convolutional Detection (2018, methodology): “In [21], a fast single-stage proposal-free detector is designed that makes use of specific height-encoded BEV input.”
MVX-Net: Multimodal VoxelNet for 3D Object Detection (2019, methodology): “Moreover, our approach consistently outperforms other recent top-performing approaches [5], [42], [28].”
Towards Robust LiDAR-based Perception in Autonomous Driving: General Black-box Adversarial Sensor Attack and Countermeasures (2020, background): “e-art models can be grouped into three classes: 1. Bird’s-eye view (BEV)-based 3D object detection. Due to the remarkable progress made in 2D image recognition tasks, a large number of existing works [7, 40, 43, 68] attempt to transform LiDAR point clouds into the 2D structure for 3D object detectio”
Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, methodology): “For voxel-based methods, point clouds are transformed into regular grids and processed with 3D sparse convolutions [17, 18] or 2D convolutions [19, 20, 21] directly.”
Pseudo-LiDAR From Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving (2018, background): “Specifically, the LiDAR signal is commonly represented as 3D point clouds [23] or viewed from the topdown “bird’s-eye view” perspective [33], and processed accordingly.”
Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection (2020, methodology): “…on point clouds into point-based (Shi, Wang, and Li 2019; Yang et al. 2019; Yilun Chen and Jia 2019; Yang et al. 2020; Shi et al. 2020a) methods and voxel-based (Zhou and Tuzel 2018; Yan, Mao, and Li 2018; Yang, Luo, and Urtasun 2018; Simon et al. 2018; Lang et al. 2019; Liu et al. 2020b) meth-ods.”
Is Pseudo-Lidar needed for Monocular 3D Object detection? (2021, methodology): “Inspired by Lidar-based methods [8, 70, 30], another line of work employs view transformation (i.e. birds-eye-view) to overcome the limitations of perspective range view, e.g. occlusion or variability in size [25, 56, 60].”
A review of deep learning methods for semantic segmentation of remote sensing imagery (2021, methodology): “…methods have demonstrated much superior performance in many traditional computer vision applications including object classification (Liu, Deng, & Yang, 2018; Shi, Yuan, Elhoseny, & Yuan, 2020), detection (Yuan, Xie, & Abouelenien, 2018), and semantic segmentation (Long, Shelhamer, & Darrell,…”