MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction

By Balakrishnan Varadarajan, Ahmed S. Hefny, A. Srivastava, Khaled S. Refaat, Nigamaa Nayakanti, Andre Cornman, K. Chen, B. Douillard, C. Lam, Drago Anguelov and 1 more. IEEE International Conference on Robotics and Automation, 2021

440 citations recorded by the source corpus.

Predicting the future behavior of road users is one of the most challenging and important problems in autonomous driving. Applying deep learning to this problem requires fusing heterogeneous world state in the form of rich perception signals and map information, and inferring highly multi-modal distributions over possible futures. In this paper, we present MultiPath++, a future prediction model that achieves state-of-the-art performance on popular benchmarks. MultiPath++ improves the MultiPath architecture [34] by revisiting many design choices. The first key design difference is a departure from dense image-based encoding of the input world state in favor of a sparse encoding of heterogeneous scene elements: MultiPath++ consumes compact and efficient polylines to describe road features, and raw agent state information directly (e.g., position, velocity, acceleration). We propose a context-aware fusion of these elements and develop a reusable multi-context gating fusion component. Second, we reconsider the choice of pre-defined static anchors, and develop a way to learn latent anchor embeddings end-to-end in the model. Lastly, we explore ensembling and output aggregation techniques—common in other ML domains—and find effective variants for our probabilistic multimodal output representation. We perform an extensive ablation on these design choices, and show that our proposed model achieves state-of-the-art performance on the Argoverse Motion Forecasting Competition [10] and the Waymo Open Dataset Motion Prediction Challenge [13].

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Safety, Risk, Reliability and Quality

View Paper: https://www.semanticscholar.org/paper/8b1602a911eb0f1753f22429952379a35c276a0f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Planning-oriented Autonomous Driving (2022, methodology): “Like most of the prior meth-ods, we model the multimodal trajectories as gaussian mixtures, and use the multi-path loss [12,90], which includes a classification score loss L cls and a negative log-likelihood loss term L nll , and λ denotes the corresponding weight: L motion = λ cls L cls + λ reg L…”
Motion Transformer with Global Intention Localization and Local Movement Refinement (2022, methodology): “VectorNet [16] is widely adopted by recent works [19, 43, 35, 47] due to its efficiency and scalability, where both road maps and agent trajectories are represented as polylines.”
Query-Centric Trajectory Prediction (2023, methodology): “(i) While the flourishing forecasting models have achieved impressive performance on trajectory prediction benchmarks [7,13,49], today’s most advanced architectures specialized for this task [37, 38, 46, 56] fail to process the heterogeneous traffic scenes efficiently.”