A General Pipeline for 3D Detection of Vehicles

By Xinxin Du, Marcelo H. Ang Jr., Sertac Karaman, Daniela Rus. 2018

156 citations recorded by the source corpus.

Autonomous driving requires 3D perception of vehicles and other objects in the in environment. Much of the current methods support 2D vehicle detection. This paper proposes a flexible pipeline to adopt any 2D detection network and fuse it with a 3D point cloud to generate 3D information with minimum changes of the 2D detection networks. To identify the 3D box, an effective model fitting algorithm is developed based on generalised car models and score maps. A two-stage convolutional neural network (CNN) is proposed to refine the detected 3D box. This pipeline is tested on the KITTI dataset using two different 2D detection networks. The 3D detection results based on these two networks are similar, demonstrating the flexibility of the proposed pipeline. The results rank second among the 3D detection algorithms, indicating its competencies in 3D detection.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Environmental Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Environmental Engineering

Citations in the corpus

Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection (2018, background): “Deep learning has been introduced to object detection in the autonomous driving settings that use cameras [1]–[4], Lidar [5]–[9], or both [10]–[15], and has set the benchmark on many popular datasets (e.”
A Survey on 3D Object Detection Methods for Autonomous Driving Applications (2019, methodology): “[67] first select the points that lie in the detection box when projected to the image plane, then use these points to perform model fitting, resulting in a preliminary 3D proposal.”
Multi-Task Multi-Sensor Fusion for 3D Object Detection (2019, methodology): “Methods like [17, 6] adopt a cascade approach by using cameras in the first stage and reasoning in LiDAR point clouds only at the second stage.”
M3D-RPN: Monocular 3D Region Proposal Network for Object Detection (2019, methodology): “LiDAR 3D Detection: The use of LiDAR data has proven to be essential input for SOTA frameworks [9, 12, 13, 24, 31, 35, 41] for 3D object detection applied to urban scenes.”
Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review (2020, unknown): “Frustum based [57] [65] [58] [59] [61] [63] [62] 1.”
3D Object Detection for Autonomous Driving: A Comprehensive Survey (2022, background): “[306] divides a viewing frustum into grid cells and applies a convolutional network on the grid cells for 3D detection; [259] proposes a novel geometric agreement search; [206] exploits the pillar representation; [67] introduces a model fitting algorithm to find the object point cloud inside each fr”
Orthographic Feature Transform for Monocular 3D Object Detection (2018, background): “To date the 3D object detection literature has been dominated by approaches which make use of rich LiDAR point clouds [37, 33, 15, 27, 5, 6, 22, 1], while the performance of image-only methods, which lack the absolute depth information of LiDAR, lags signiﬁcantly behind.”
SECOND: Sparsely Embedded Convolutional Detection (2018, unknown): “Point cloud data are converted into a 2D bird’s eye view image [8] or are projected onto an image [9,10].”
From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network (2019, background): “W ITH the surging demand from autonomous driving and robotics, increasing attention has been paid to 3D object detection [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13].”
EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection (2020, methodology): “10 - - - PC-CNN [5] L+I 85.”
Deep Continuous Fusion for Multi-sensor 3D Object Detection (2018, methodology): “Other approaches [6,18,8] apply 2D convolutional networks on both camera image and LIDAR bird’s eye view (BEV) representations, and fuse them at the intermediate region-wise convolutional feature map via feature concatenation.”
Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks (2021, methodology): “By contrast, production high-level AD systems such as Waymo, Pony.ai, and Baidu Apollo, typically adopt a Multi-Sensor Fusion (MSF) based design [14]–[17], which fuses the results from different perception sources, e.g., LiDAR and camera, to achieve overall higher accuracy and robustness [18]–[26].”
Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges (2019, methodology): “some papers including [97], [101]–[104], [106], [112], [114]”