Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection

By Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, Xiangyu Zhang. IEEE International Conference on Computer Vision, 2023

444 citations recorded by the source corpus.

In this paper, we propose a long-sequence modeling framework, named StreamPETR, for multi-view 3D object detection. Built upon the sparse query design in the PETR series, we systematically develop an object-centric temporal mechanism. The model is performed in an online manner and the long-term historical information is propagated through object queries frame by frame. Besides, we introduce a motion-aware layer normalization to model the movement of the objects. StreamPETR achieves significant performance improvements only with negligible computation cost, compared to the single-frame baseline. On the standard nuScenes benchmark, it is the first online multi-view method that achieves comparable performance (67.6% NDS & 65.3% AMOTA) with lidar-based methods. The lightweight version realizes 45.0% mAP and 31.7 FPS, outperforming the state-of-the-art method (SOLOFusion) by 2.3% mAP and 1.8× faster FPS. Code has been available at https://github.com/exiawsh/StreamPETR.git.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/73385e2aac9890073669759dfde8800b5704ab6e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Jian‐Yuan Sun
2. Tiancai Wang
3. Yingfei Liu
4. Zhaoxiang Zhang
5. Junjie Yan
6. Qian Zhang
7. X. Zhang
8. Bencheng Liao

In Computer Vision and Pattern Recognition

1. Xinggang Wang
2. Luc Van Gool
3. Junjie Yan
4. Wenyu Liu
5. Dacheng Tao
6. Xiaogang Wang
7. Qian Zhang
8. Chunhua Shen

Citations in the corpus

• VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning (2024, background): “Subsequent works [25, 48] continuously improve performance in downstream tasks by optimizing temporal modeling and BEV transformation strategies.”
• Driving Into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving (2023, methodology): “Additionally, we evaluate 3D object detection [62] and online map construction [38].”
• PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images (2022, methodology): “M 2 BEV [40] further uses the camera parameters to project the features extracted from backbone to the 3D ego-car co-ordinate to generate the BEV representation.”