PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images

By Yingfei Liu, Junjie Yan, Fan Jia, Shuailin Li, Qi Gao, Tiancai Wang, X. Zhang, Jian‐Yuan Sun. IEEE International Conference on Computer Vision, 2022

541 citations recorded by the source corpus.

In this paper, we propose PETRv2, a unified framework for 3D perception from multi-view images. Based on PETR [25], PETRv2 explores the effectiveness of temporal modeling, which utilizes the temporal information of previous frames to boost 3D object detection. More specifically, we extend the 3D position embedding (3D PE) in PETR for temporal modeling. The 3D PE achieves the temporal alignment on object position of different frames. To support for multi-task learning (e.g., BEV segmentation and 3D lane detection), PETRv2 provides a simple yet effective solution by introducing task-specific queries, which are initialized under different spaces. PETRv2 achieves state-of-the-art performance on 3D object detection, BEV segmentation and 3D lane detection. Detailed robustness analysis is also conducted on PETR framework. Code is available at https://github.com/megvii-research/PETR.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/03f08915ce10cabea051e654dd72ef93588202f6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Xizhou Zhu
2. Song Han
3. Zhijian Liu
4. Tiancai Wang
5. Yingfei Liu
6. Yu Qiao
7. Gao Huang
8. Zhaoxiang Zhang

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Dacheng Tao
3. Yu Qiao
4. Xiaogang Wang
5. Song Han
6. Chunhua Shen
7. Yi Yang
8. Alan L. Yuille

Citations in the corpus

• Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving (2023, unknown): “More recently, vision-based 3D object detection has gained more attention due to its low cost and rich semantic content [41, 46, 48, 22, 26, 33, 28, 13, 27, 15, 21, 31, 25].”
• BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation (2022, background): “BEVDet4D [19], BEVFormer [25] and PETRv2 [31] exploit temporal cues in multi-camera 3D object detection, achieving significant”
• BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision (2022, methodology): “BEVStereo [14] and STS [37] both adopted a temporal stereo paradigm for better depth estimation.”
• DETRs with Hybrid Matching (2022, methodology): “We choose two representative open-sourced SOTA methods, including 3 DETR [46] and PETRv 2 [36, 37], to verify the generalization ability of our approach on 3 D object detection.”
• Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection (2023, methodology): “The temporal function φ is usually achieved by the spatial cross-attention [31, 29, 34]: The cross-attention between object query and multi-frame perspective view requires repeated feature aggregation.”