RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving

By Peixuan Li, Huaici Zhao, Pengfei Liu, Feidao Cao. European Conference on Computer Vision, 2020

371 citations recorded by the source corpus.

In this work, we propose an efficient and accurate monocular 3D detection framework in single shot. Most successful 3D detectors take the projection constraint from the 3D bounding box to the 2D box as an important component. Four edges of a 2D box provide only four constraints and the performance deteriorates dramatically with the small error of the 2D detector. Different from these approaches, our method predicts the nine perspective keypoints of a 3D bounding box in image space, and then utilize the geometric relationship of 3D and 2D perspectives to recover the dimension, location, and orientation in 3D space. In this method, the properties of the object can be predicted stably even when the estimation of keypoints is very noisy, which enables us to obtain fast detection speed with a small architecture. Training our method only uses the 3D properties of the object without the need for external networks or supervision data. Our method is the first real-time system for monocular image 3D detection while achieves state-of-the-art performance on the KITTI benchmark. Code will be released at this https URL.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/3a71d497ca1b33cfac338dc7a818fe69ad2f528c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Probabilistic and Geometric Depth: Detecting Objects in Perspective (2021, methodology): “Our framework based on the simple adapted FCOS3D achieves much better performance than others, especially considering M3D-RPN [13] and RTM3D [14] adopt stronger backbone and data augmentation.”
FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection (2021, methodology): “Another stream of methods based on redundant 3D information [13, 16] predicts extra keypoints for optimized results ultimately.”
MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships (2020, result): “Very recently, several concurrent works [24, 21] also adopt a keypoint detection strategy similar to our work.”
Multimodal Virtual Point 3D Detection (2021, background): “On state-of-the-art 3D detection benchmarks [2, 12], state-of-the-art monocular methods [26, 41] achieve about half the mAP detection accuracy, of standard Lidar based baselines [60].”
Categorical Depth Distribution Network for Monocular 3D Object Detection (2021, background): “For example, object keypoints can be estimated on the image plane, in order to assist in 3D box construction using known geometry [33, 29].”
Objects are Different: Flexible Monocular 3D Object Detection (2021, methodology): “The penalty-reduced focal loss [26] is utilized for heatmap prediction as in [10, 25, 28].”
I3DOL: Incremental 3D Object Learning without Catastrophic Forgetting (2020, background): “e superiority of our I3DOL framework. Introduction Object recognition technology has achieved remarkable developments in great quantiﬁes of research ﬁelds, e.g., autonomous driving (Behl et al. 2017; Li et al. 2020), intelligent robotics (Stria and Hlavac 2018; Wei, Deng, and Yang´ 2020; Chen et al.”
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection (2021, background): “RTM3D (Li et al., 2020) performs coarse keypoints detection to localize 3d objects.”