BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision

By Chenyu Yang, Yuntao Chen, Haofei Tian, Chenxin Tao, Xizhou Zhu, Zhaoxiang Zhang, Gao Huang, Hongyang Li, Yu Qiao, Lewei Lu and 2 more. Computer Vision and Pattern Recognition, 2022

460 citations recorded by the source corpus.

We present a novel bird's-eye-view (BEV) detector with perspective supervision, which converges faster and bet-suits modern image backbones. Existing state-of-the-art BEV detectors are often tied to certain depth pretrained backbones like Vo Vn et, hindering the synergy between booming image backbones and BEV detectors. To address this limitation, we prioritize easing the optimization of BEV detectors by introducing perspective view supervision. To this end, we propose a two-stage BEV detector; where proposals from the perspective head are fed into the bird’ s-eye-view head for final predictions. To evaluate the effectiveness of our model, we conduct extensive ablation studies focusing on the form of supervision and the gener-ality of the proposed detector. The proposed method is ver-ified with a wide spectrum of traditional and modern image backbones and achieves new SoTA results on the large-scale nuScenes dataset. The code shall be released soon.

Subfields: Computer Vision and Pattern Recognition, Media Technology

View Paper: https://www.semanticscholar.org/paper/968fc508273f20a8c9cfc511a3d372f54198592b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Wenyu Liu
2. Qian Zhang
3. Xinggang Wang
4. Tiancai Wang
5. Yingfei Liu
6. Xiangyu Zhang
7. Shihao Wang
8. Ying Li

In Media Technology

1. Qian Zhang
2. Wenyu Liu
3. Xinggang Wang
4. Tiancai Wang

Citations in the corpus

• Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection (2023, methodology): “Following BEVFormerv2 [50], the ViT-Large [8] is pre-trained on Objects365 [41] and COCO [28] dataset.”
• VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning (2024, background): “BEVFormer [26, 52], on the other hand, avoids explicit depth prediction by designing spatial and temporal attention mechanisms, and achieves impressive detection performance.”