BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection

By Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jian‐Yuan Sun, Zeming Li. AAAI Conference on Artificial Intelligence, 2022

1,027 citations recorded by the source corpus.

In this research, we propose a new 3D object detector with a trustworthy depth estimation, dubbed BEVDepth, for camera-based Bird's-Eye-View~(BEV) 3D object detection. Our work is based on a key observation -- depth estimation in recent approaches is surprisingly inadequate given the fact that depth is essential to camera 3D detection. Our BEVDepth resolves this by leveraging explicit depth supervision. A camera-awareness depth estimation module is also introduced to facilitate the depth predicting capability. Besides, we design a novel Depth Refinement Module to counter the side effects carried by imprecise feature unprojection. Aided by customized Efficient Voxel Pooling and multi-frame mechanism, BEVDepth achieves the new state-of-the-art 60.9% NDS on the challenging nuScenes test set while maintaining high efficiency. For the first time, the NDS score of a camera model reaches 60%. Codes have been released.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/234f0122e0edccba5c91763e800c2f02fe8ae4fe

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Media Technology

Citations in the corpus

Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection (2023, methodology): “Then a temporal fusion function φ (concatenation [16, 25] or deformable attention [27]) can be applied for intermediate temporal representation F̃ t bev .”
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving (2023, methodology): “Better viewed when zoomed in. camera images as inputs, it has demonstrated competitive performance in various 3D perception tasks including depth estimation [17, 58], 3D object detection [30, 27, 36, 34, 20], and semantic map construction [66, 19, 1].”
Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction (2023, methodology): “Though they have dominated the 3D segmentation task [47], they still lag behind BEV-based methods on the 3D detection performance [27].”
NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario (2023, background): “As the increasing maturity of 3D object detection [26, 28, 42] and tracking [6, 42], the challenges of autonomous driving systems are shifting from traditional perception to interpretability and trust-worthiness.”
MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction (2022, methodology): “…is compatible with other transformation methods and maintains stable performance, e.g. , CVT (Zhou & Kr¨ahenb¨uhl, 2022), LSS (Philion & Fidler, 2020; Liu et al., 2022c; Li et al., 2022b; Huang et al., 2021), Deformable Attention (Li et al., 2022c; Zhu et al., 2021) and IPM (Mallot et al., 1991).”
OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction (2023, methodology): “Considering the dimension gap between the 2D image input and the 3D prediction, recent studies for vision-based 3D perception first construct the BEV feature representations and then perform various downstream tasks on the BEV space [19, 28, 30, 37, 58, 38, 60, 40, 18, 1, 42].”
Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-Shot Metric Depth and Surface Normal Estimation (2024, neutral): “These unique attributes render both depth and surface normals indispensable in various computer vision applications, including 3D reconstruction [1], [2], [3], neural rendering (NeRF) [4], [5], [6], [7], autonomous driving [8], [9], [10], and robotics [11], [12], [13].”
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision (2022, methodology): “As for BEV detectors, BEVDepth [15] utilized Li-DAR point clouds to supervise its intermediate depth network.”
VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning (2024, background): “Subsequent works [25, 48] continuously improve performance in downstream tasks by optimizing temporal modeling and BEV transformation strategies.”
Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving (2023, positive): “More recently, vision-based 3D object detection has gained more attention due to its low cost and rich semantic content [41, 46, 48, 22, 26, 33, 28, 13, 27, 15, 21, 31, 25].”