BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View

By Junjie Huang, Guan Huang, Zheng Zhu, Dalong Du. arXiv.org, 2021

1,076 citations recorded by the source corpus.

Autonomous driving perceives its surroundings for decision making, which is one of the most complex scenarios in visual perception. The success of paradigm innovation in solving the 2D object detection task inspires us to seek an elegant, feasible, and scalable paradigm for fundamentally pushing the performance boundary in this area. To this end, we contribute the BEVDet paradigm in this paper. BEVDet performs 3D object detection in Bird-Eye-View (BEV), where most target values are defined and route planning can be handily performed. We merely reuse existing modules to build its framework but substantially develop its performance by constructing an exclusive data augmentation strategy and upgrading the Non-Maximum Suppression strategy. In the experiment, BEVDet offers an excellent trade-off between accuracy and time-efficiency. As a fast version, BEVDet-Tiny scores 31.2% mAP and 39.2% NDS on the nuScenes val set. It is comparable with FCOS3D, but requires just 11% computational budget of 215.3 GFLOPs and runs 9.2 times faster at 15.6 FPS. Another high-precision version dubbed BEVDet-Base scores 39.3% mAP and 47.2% NDS, significantly exceeding all published results. With a comparable inference speed, it surpasses FCOS3D by a large margin of +9.8% mAP and +10.0% NDS. The source code is publicly available for further research at https://github.com/HuangJunJie2017/BEVDet .

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/56a7a5bc55a5af2b1a78a0c18b16a5af6f5d3670

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario (2023, methodology): “In addition to the dataset, we also develop baseline models us-ing the existing 3D perception [16, 20, 42] and visual question answering [1, 43] techniques.”
Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction (2023, methodology): “Recent studies in BEV-based perception focus on how to effectively transform features from the image space to the BEV space [20, 26, 27, 35, 36, 48].”
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision (2022, background): “BEVDet [10] introduced rich image-level and BEV-level augmentations for training.”
OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction (2023, methodology): “Considering the dimension gap between the 2D image input and the 3D prediction, recent studies for vision-based 3D perception first construct the BEV feature representations and then perform various downstream tasks on the BEV space [19, 28, 30, 37, 58, 38, 60, 40, 18, 1, 42].”
PETR: Position Embedding Transformation for Multi-View 3D Object Detection (2022, methodology): “Since the DETR3D [51] and BEVDet [18] follow different settings on the image size and backbone initialization, we individually compare the PETR with them for fair comparison.”
BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework (2022, methodology): “Methods [37, 15, 52] are also inspired by LSS [32] and refer to the LiDAR for the supervision on depth prediction.”
V2V4Real: A Real-World Large-Scale Dataset for Vehicle-to-Vehicle Cooperative Perception (2023, background): “(1) Camera-based detection denotes approaches that detect 3D objects from a single or multiple RGB images [16, 32, 33, 35, 42].”
MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction (2022, methodology): “And for fair comparison with IPM and LSS, GKT and Deformable Attention both adopt one-layer conﬁguration.”
DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving (2024, neutral): “End-to-end autonomous driving has gained signiﬁcant attention in recent years due to advancements in perception models (detection [5, 17, 24, 42], tracking [54–56], online mapping [ 27, 28, 32], etc .)”
Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection (2023, methodology): “For image and BEV data augmentation, we adopt the same methods as PETR [18, 30].”
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving (2023, methodology): “camera images as inputs, it has demonstrated competitive performance in various 3D perception tasks including depth estimation [18, 57], 3D object detection [29, 28, 35, 33, 21], and semantic map construction [65, 20, 1].”
BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation (2022, background): “They approximate the view transformer by assuming uniform depth distribution [41] or truncating the points within each BEV grid [40].”
Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving (2023, positive): “We extend two main-stream BEV models – BEVDet [14] and BEVFormer [22] to the 3D occupancy prediction task.”
Unifying Voxel-based Representation with Transformer for 3D Object Detection (2022, methodology): “Different from methods [5, 6] for BEV space, we preserve the 3D voxel space without collapsing in Z axis to avoid the aforementioned semantic ambiguity and enable further interactions.”
PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, methodology): “, BEVDet [11]) explicitly transform the multi-view features into bird-eye-view (BEV) representation by LSS [30].”
Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignment (2022, background): “We report the results for two additional problems: multi-view 3D object detection [24, 32, 37, 38] and instance segmentation [9, 30], to further demonstrate the effectiveness.”
BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection (2022, methodology): “Thus, the existing paradigms [40, 41, 42, 22, 15] perform relatively poorly in predicting the time-relevant targets like velocity than the LiDAR-based [46] or radar-based [27] methods.”
3D Object Detection for Autonomous Driving: A Comprehensive Survey (2022, methodology): “4 - BEVDet [106] Camera Multi-View 2022 - 42.”