Complex-YOLO: An Euler-Region-Proposal for Real-Time 3D Object Detection on Point Clouds

By Martin Simon, Stefan Milz, Karl Amende, Horst-Michael Groß. 2018

360 citations recorded by the source corpus.

Lidar based 3D object detection is inevitable for autonomous driving, because it directly links to environmental understanding and therefore builds the base for prediction and motion planning. The capacity of inferencing highly sparse 3D data in real-time is an ill-posed problem for lots of other application areas besides automated vehicles, e.g. augmented reality, personal robotics or industrial automation. We introduce Complex-YOLO, a state of the art real-time 3D object detection network on point clouds only. In this work, we describe a network that expands YOLOv2, a fast 2D standard object detector for RGB images, by a specific complex regression strategy to estimate multi-class 3D boxes in Cartesian space. Thus, we propose a specific Euler-Region-Proposal Network (E-RPN) to estimate the pose of the object by adding an imaginary and a real fraction to the regression network. This ends up in a closed complex space and avoids singularities, which occur by single angle estimations. The E-RPN supports to generalize well during training. Our experiments on the KITTI benchmark suite show that we outperform current leading methods for 3D object detection specifically in terms of efficiency. We achieve state of the art results for cars, pedestrians and cyclists by being more than five times faster than the fastest competitor. Further, our model is capable of estimating all eight KITTI-classes, including Vans, Trucks or sitting pedestrians simultaneously with high accuracy.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Environmental Engineering, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Environmental Engineering

In 3D Vision & Pose

Citations in the corpus

WoodScape: A Multi-Task, Multi-Camera Fisheye Dataset for Autonomous Driving (2019, methodology): “1°) 71 Visual SLAM LSD SLAM [15] Relocalization (%) 61 3D Bounding Box Detection - Complex YOLO [50] Metric for Training AP (%) AOS (%) Runtime (ms)”
Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection (2020, methodology): “2020a) methods and voxel-based (Zhou and Tuzel 2018; Yan, Mao, and Li 2018; Yang, Luo, and Urtasun 2018; Simon et al. 2018; Lang et al. 2019; Liu et al. 2020b) methods.”
From Points to Parts: 3D Object Detection From Point Cloud With Part-Aware and Part-Aggregation Network (2019, background): “W ITH the surging demand from autonomous driving and robotics, increasing attention has been paid to 3D object detection [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13].”
PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation (2020, background): “Recently, variants of the initial BEV attempt to encode each pixel in the BEV with rich different heights [38], reﬂection [28] and even learned representations [16].”
End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds (2019, methodology): “r both 2D and 3D object detection. The transformation process is usually hand-crafted, some representative works include Vote3D [17], Vote3Deep [18], 3DFCN [19], AVOD [20], PIXOR [1] and Complex YOLO [21]. VoxelNet by Zhou et al. [2] divides the point cloud into a 3D voxel grid (i.e. voxels) and use”
FVNet: 3D Front-View Proposal Generation for Real-Time Object Detection from Point Clouds (2019, methodology): “To obtain a more compact representation, some methods transform point clouds into 2D maps by projecting point clouds onto the ground plane (Bird’s Eye View) [12]–[14] or depth maps (Front View) [15], [16], and then apply 2D convolutional neural networks on the transformed maps to execute the detecti”
Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds (2022, neutral): “Due to the unstructured and orderless nature of 3D point clouds, early works usually first convert the raw point clouds into intermediate regular representation, including projecting the 3D point clouds into 2D images from birdseye-view or frontal view [1, 17, 18, 41, 42, 50, 60], or transformed int”
An End-to-End Transformer Model for 3D Object Detection (2021, background): “3D Object Detection is a well studied research area where methods predict three dimensional bounding boxes from 3D input data [23, 41, 43, 52, 54, 55, 70, 72, 93].”
Center-based 3D Object Detection and Tracking (2020, background): “These marked differences between 2D and 3D detection, made a transfer of ideas between the two domain harder [27, 28, 39].”
Structure Aware Single-Stage 3D Object Detection From Point Cloud (2020, methodology): “We evaluate our proposed structure-aware single-stage detector (SA-SSD) on the KITTI 3D/BEV object detection benchmark [4].”
A Survey of End-to-End Driving: Architectures and Training Methods (2020, background): “A further option is to project the point cloud to a 2D grid in top-down view (bird-eye view (BEV)) [94], [95].”
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review (2020, methodology): “DL architectures on 3D object detection/localization with three different data representations: point cloud based networks represented by VoteNet [42], voxel-based networks represented by VoxelNet [13], view-based networks represented by ComplexYOLO [116].”
Cooperative Perception for 3D Object Detection in Driving Scenarios Using Infrastructure Sensors (2019, methodology): “object detection. After the projection and representation of the points into a ﬁxed size input tensor, convolutional layers are applied to generate the ﬁnal 3D bounding boxes on a single forward pass [18]. Both projection techniques can result in loss of information due to space quantization and the”