MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction

By Yuning Chai, Benjamin Sapp, Mayank Bansal, Dragomir Anguelov. Conference on Robot Learning, 2019

841 citations recorded by the source corpus.

Predicting human behavior is a difficult and crucial task required for motion planning. It is challenging in large part due to the highly uncertain and multi-modal set of possible outcomes in real-world domains such as autonomous driving. Beyond single MAP trajectory prediction, obtaining an accurate probability distribution of the future is an area of active interest. We present MultiPath, which leverages a fixed set of future state-sequence anchors that correspond to modes of the trajectory distribution. At inference, our model predicts a discrete distribution over the anchors and, for each anchor, regresses offsets from anchor waypoints along with uncertainties, yielding a Gaussian mixture at each time step. Our model is efficient, requiring only one forward inference pass to obtain multi-modal future distributions, and the output is parametric, allowing compact communication and analytical probabilistic queries. We show on several datasets that our model achieves more accurate predictions, and compared to sampling baselines, does so with an order of magnitude fewer trajectories.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/705935dcba4a4922b2d7c15741acef570fb37b75

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation (2020, methodology): “More recently, many learning-based approaches are proposed [5, 6, 10, 15]; they offer the benefit of having probabilistic interpretations of different behavior hypotheses, but require building a representation to encode the map and trajectory information.”
VAD: Vectorized Scene Representation for Efficient Autonomous Driving (2023, neutral): “3.4).”
MotionDiffuser: Controllable Multi-Agent Motion Prediction Using Diffusion (2023, methodology): “For this, MultiPath [40] uses static anchors, and MultiPath++ [48], Wayformer [31], SceneTransformer [32] use learned anchors, and DenseTNT [13] uses goal-based predictions.”
Multiple Futures Prediction (2019, background): “DESIRE SocialGAN R2P2-MA ESP[30] ESP MultiPath MFP7 [21] [30] no LIDAR [8] Town01 2.599 1.464 0.843 1.213 0.716 0.68 0.584 test ±0.029 ±0.028 ±0.014 ±0.019 ±0.012 ±0.008 Town02 2.422 1.141 0.770 1.102 0.675 0.69 0.512 test ±0.017 ±0.015 ±0.008 ±0.011 ±0.007 ±0.004 Table 4: Test performance (minMSD w”
TNT: Target-driveN Trajectory Prediction (2020, methodology): “Most related to TNT are works that discretize the output space as intents [32] or with anchors [33, 34].”
V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction (2020, background): “reason in 3D and intelligently fuse multi-sensor data (e.g., images, LiDAR, maps) [28,29]. Motion forecasting algorithm performance has been further improved by building good multimodal distributions [4,6,12,19] that capture diverse actor behaviour and by modelling actor interactions [3,25,36,37]. R”
Planning-oriented Autonomous Driving (2022, methodology): “Meanwhile, some motion forecasting methods implicitly include the planning task by producing their future trajectories simultaneously [12,45,70].”
Wayformer: Motion Forecasting via Simple & Efficient Attention Networks (2022, methodology): “During training, we follow [8], [45] in decomposing the loss into separate classification and regression losses.”
Human motion trajectory prediction: a survey (2019, background): “Recently many authors have proposed solutions to this end (Chai et al. 2019; Zhao et al. 2019; Hong et al. 2019; Cui et al. 2019).”
HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction (2022, methodology): “The past trajectories of agents are either rasterized as additional image channels [9, 12] or processed by temporal models like RNNs [2, 41, 42].”
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting (2021, methodology): “On the nuScenes dataset, we evaluate our approach against state-of-the-art vehicle prediction meth-ods – Trajectron++ [45], MTP [8], MultiPath [5] Net [39], DSF-AF [33], and DLow-AF [59].”
CoverNet: Multimodal Behavior Prediction Using Trajectory Sets (2019, methodology): “Multimodal approaches output either a distribution over multiple trajectories [7, 13, 20, 14] or a spatial-temporal occupancy map [20, 28, 34].”
MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction (2021, methodology): “Gaussian Mixture Models ( GMM s) are a popular choice for this purpose due to their compact parameterized form; mode collapse is addressed through training tricks [14, 50] or the use of trajectory anchors [45].”
Query-Centric Trajectory Prediction (2023, methodology): “Early work rasterizes world states as multichannel images and employs classic convolutional neural networks for learning [5,6,10,21].”
MP3: A Unified Model to Map, Perceive, Predict and Plan (2021, background): “Many accurate LiDAR-based object detectors have been proposed [25, 34, 52, 55] to localize dynamic obstacles followed by a motion forecasting stage [7, 10, 30, 41, 47] to predict the future state of each object.”
GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation (2021, methodology): “Epistemic uncertainty outlines some knowledge that can’t 21 be known by the observer at prediction time: what is the car destination, will it choose to overtake 22 the car by the left or stay behind ? Recent methods use multimodal outputs based on anchors [3, 4], 23 predefined learning heads [5] or ”
Multimodal Motion Prediction with Stacked Transformers (2021, methodology): “Finally, the trajectory is directly generated by model [12] or based on prior knowledge [11, 33, 5].”
TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors (2021, background): “Since the future is inherently uncertain, [17, 14] output multiple trajectories per actor.”
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset (2021, background): “Motion forecasting has received increasing attention as a critical requirement for planning in autonomous driving systems [8, 14, 40, 36, 28, 34].”