The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs

By Boris Ivanovic, Marco Pavone. IEEE International Conference on Computer Vision, 2018

488 citations recorded by the source corpus.

Developing safe human-robot interaction systems is a necessary step towards the widespread integration of autonomous agents in society. A key component of such systems is the ability to reason about the many potential futures (e.g. trajectories) of other agents in the scene. Towards this end, we present the Trajectron, a graph-structured model that predicts many potential future trajectories of multiple agents simultaneously in both highly dynamic and multimodal scenarios (i.e. where the number of agents in the scene is time-varying and there are many possible highly-distinct futures for each agent). It combines tools from recurrent sequence modeling and variational deep generative modeling to produce a distribution of future trajectories for each agent in a scene. We demonstrate the performance of our model on several datasets, obtaining state-of-the-art results on standard trajectory prediction metrics as well as introducing a new metric for comparing models that output distributions.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/05b16031728e1a218ac95a1cef914df73c4204ae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion (2022, methodology): “In addition, the spatialtemporal graph model is applied to jointly model the temporal clues and social interactions [15, 16, 30, 38, 44, 50].”
SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction (2021, methodology): “To model the interactions between pedestrians, extensive works [31, 2, 11, 23, 19, 32, 46] have been done in the past few years, in which the weighting-by-distance methods [31, 2, 11, 32] and the attention-based meth-ods [23, 19, 46, 8, 17, 18] have achieved the state-of-the-art results in…”
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting (2021, methodology): “…recent trajectory prediction methods start to model the trajectory distribution with deep generative models [21, 13, 40] such as conditional variational autoencoders (CVAEs) [27, 57, 19, 46, 51, 45], generative adversarial networks (GANs) [15, 44, 25, 62], and normalizing ﬂows (NFs) [41, 42, 14].”
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving (2022, background): “Attention has been explored in the context of driving for lane changing [65], object detection [31], [66], [67], [68], motion forecasting [31], [69], [70], [71], [72], [73], [74], [75], [76], [77], driver attention prediction [78], [79] and recently also for end-to-end driving [39], [41].”
Human motion trajectory prediction: a survey (2019, methodology): “Other approaches use RNN as models of spatio-temporal graphs for problems that require both spatial and temporal reasoning (Jain et al. 2016; Vemula et al. 2018; Huang et al. 2019; Dai et al. 2019; Ivanovic and Pavone 2019; Eiffert and Sukkarieh 2019).”
Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction (2020, methodology): “Crowd trajectory prediction is of fundamental importance to both the computer vision [1,16,53,21,22] and robotics [34,33] community.”
Transformer Networks for Trajectory Forecasting (2020, background): “the ground truth, averaging the discrepancy at each time step; Final Average Displacement (FAD, equivalently Final Displacement Error FDE [8]), to check the goodness of the prediction at the last time step.”
Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data (2020, methodology): “S-GAN [13] SoPhie [40] Trajectron [18] S-BiGAT [27] Ours (Full) Ours+ ∫ (Full)”
From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting (2020, background): “Hence, we report the kernel density estimate-based negative log-likelihood (KDE-NLL) metric in the same fashion as [17, 36].”