Query-Centric Trajectory Prediction

By Zikang Zhou, Jianping Wang, Yung-Hui Li, Yu-Kai Huang. Computer Vision and Pattern Recognition, 2023

376 citations recorded by the source corpus.

Predicting the future trajectories of surrounding agents is essential for autonomous vehicles to operate safely. This paper presents QCNet, a modeling framework toward pushing the boundaries of trajectory prediction. First, we identify that the agent-centric modeling scheme used by existing approaches requires re-normalizing and re-encoding the input whenever the observation window slides forward, leading to redundant computations during online prediction. To overcome this limitation and achieve faster inference, we introduce a query-centric paradigm for scene encoding, which enables the reuse of past computations by learning representations independent of the global spacetime coordinate system. Sharing the invariant scene features among all target agents further allows the parallelism of multi-agent trajectory decoding. Second, even given rich encodings of the scene, existing decoding strategies struggle to capture the multimodality inherent in agents' future behavior, especially when the prediction horizon is long. To tackle this challenge, we first employ anchor-free queries to generate trajectory proposals in a recurrent fashion, which allows the model to utilize different scene contexts when decoding waypoints at different horizons. A refinement module then takes the trajectory proposals as anchors and leverages anchor-based queries to refine the trajectories further. By supplying adaptive and high-quality anchors to the refinement module, our query-based decoder can better deal with the multimodality in the output of trajectory prediction. Our approach ranks 1st on Argoverse 1 and Argoverse 2 motion forecasting benchmarks, outperforming all methods on all main metrics by a large margin. Meanwhile, our model can achieve streaming scene encoding and parallel multi-agent decoding thanks to the query-centric design ethos.

Subfields: Automotive Engineering, Safety, Risk, Reliability and Quality, Signal Processing

View Paper: https://www.semanticscholar.org/paper/063fa6d4d525bbe9db58c7f3431b9aaace911afb

Citations in the corpus

• Low-latency automotive vision with event cameras (2024, neutral): “Moreover, this lies within the tolerance limit of the current agent-forecasting methods 10–12 that are currently finding their way into commercial patents 13 , in which we see displacement errors in prediction of the order of 0.6 m, more than one order of magnitude higher than the worst-case error…”