Wayformer: Motion Forecasting via Simple & Efficient Attention Networks

By Nigamaa Nayakanti, Rami Al-Rfou, Aurick Zhou, Kratarth Goel, Khaled S. Refaat, Benjamin Sapp. IEEE International Conference on Robotics and Automation, 2022

449 citations recorded by the source corpus.

Motion forecasting for autonomous driving is a challenging task because complex driving scenarios involve a heterogeneous mix of static and dynamic inputs. It is an open problem how best to represent and fuse information about road geometry, lane connectivity, time-varying traffic light state, and history of a dynamic set of agents and their interactions into an effective encoding. To model this diverse set of input features, many approaches proposed to design an equally complex system with a diverse set of modality specific modules. This results in systems that are difficult to scale, extend, or tune in rigorous ways to trade off quality and efficiency. In this paper, we present Wayformer, a family of simple and homogeneous attention based architectures for motion forecasting. Wayformer offers a compact model description consisting of an attention based scene encoder and a decoder. In the scene encoder we study the choice of early, late and hierarchical fusion of input modalities. For each fusion type we explore strategies to trade off efficiency and quality via factorized attention or latent query attention. We show that early fusion, despite its simplicity, is not only modality agnostic but also achieves state-of-the-art results on both Waymo Open Motion Dataset (WOMD) and Argoverse leaderboards, demonstrating the effectiveness of our design philosophy.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Safety, Risk, Reliability and Quality

View Paper: https://www.semanticscholar.org/paper/44f6612c238297304331d6fe6aa4b4f909f1c6f0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Safety, Risk, Reliability and Quality

Citations in the corpus

Query-Centric Trajectory Prediction (2023, methodology): “(i) While the flourishing forecasting models have achieved impressive performance on trajectory prediction benchmarks [7,13,49], today’s most advanced architectures specialized for this task [37, 38, 46, 56] fail to process the heterogeneous traffic scenes efficiently.”
Planning-oriented Autonomous Driving (2022, background): “Recent studies have proven the effectiveness of transformer structure on the motion task [24,25,35,39,40,48,55], inspired by which we propose MotionFormer in the end-toend setting.”
Parting with Misconceptions about Learning-based Vehicle Motion Planning (2023, result): “Contrary to the community’s growing preference for graph- and vector-based scene representations in prediction and planning [66, 11, 67, 68], these results show no clear advantage in adopting these methods for the closed-loop task, with the raster-based PlanCNN also offering a lower runtime.”
DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models (2024, background): “Specifically, 3D perception [28, 30, 39, 45] is limited to detecting and tracking familiar objects, omitting rare objects and their unique attributes; motion prediction [16, 18, 33, 35, 56] and planning [2, 31, 38] focus on trajectory-level actions, often neglecting the decision-level interactions…”
NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration (2023, methodology): “Visual goal-conditioned policies: To train goal-conditioned policies for visual inputs, we follow a large body of prior work on training high-capacity policies based on the Trans-former architecture [3, 36–38].”