Probabilistic and Geometric Depth: Detecting Objects in Perspective

By Tai Wang, Xinge Zhu, Jiangmiao Pang, Dahua Lin. Conference on Robot Learning, 2021

366 citations recorded by the source corpus.

3D object detection is an important capability needed in various practical applications such as driver assistance systems. Monocular 3D detection, as a representative general setting among image-based approaches, provides a more economical solution than conventional settings relying on LiDARs but still yields unsatisfactory results. This paper first presents a systematic study on this problem. We observe that the current monocular 3D detection can be simplified as an instance depth estimation problem: The inaccurate instance depth blocks all the other 3D attribute predictions from improving the overall detection performance. Moreover, recent methods directly estimate the depth based on isolated instances or pixels while ignoring the geometric relations across different objects. To this end, we construct geometric relation graphs across predicted objects and use the graph to facilitate depth estimation. As the preliminary depth estimation of each instance is usually inaccurate in this ill-posed setting, we incorporate a probabilistic representation to capture the uncertainty. It provides an important indicator to identify confident predictions and further guide the depth propagation. Despite the simplicity of the basic idea, our method, PGD, obtains significant improvements on KITTI and nuScenes benchmarks, achieving 1st place out of all monocular vision-only methods while still maintaining real-time efficiency. Code and models will be released at https://github.com/open-mmlab/mmdetection3d.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9db3a6099fc14de22a26bf16b9b705891e30fe34

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving (2023, methodology): “Other methods implicitly learn 3D features without producing explicit depth maps [53, 29, 55, 34, 52, 63].”
BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, methodology): “With the development of autonomous driving datasets that have more sensors, like nuScenes[2] and Waymo [44], there exists a trend of developing methods [50, 51, 53] that take multi-view images as input and found to be signiﬁcantly superior to monocular methods.”
BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View (2021, methodology): “For monocular paradigms like FCOS3D [42] and PGD [43], the inference speeds are divided by a factor of 6 (i.”
BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation (2022, background): “FCOS3D [52] extends an image detector [48] with additional 3D regression branches, which is later improved by [53, 4] in depth modeling.”
BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection (2022, methodology): “The pooled BEV features from different frames are directly concatenated and fed to following tasks. nuScenes val set We compare the proposed BEVDepth with other state-of-the-art methods like FCOS3D, DETR3D, BEVDet, PETR, BEVDet4D and BEVFormer on nuScenes val set.”
PETR: Position Embedding Transformation for Multi-View 3D Object Detection (2022, methodology): “However, PGD [48] achieves relatively lower mATE because of the explicit depth supervision.”
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, neutral): “Previous works [2,6,13–15, 29, 35 , 37, 38 ] perform 3D object object”
BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers (2022, background): “The most straightforward solution is based on the monocular frameworks [45, 44, 31, 35, 3] and cross-camera post-processing.”