TNT: Target-driveN Trajectory Prediction

By Hang Zhao, Jiyang Gao, Tian Lan, Chen Sun, Benjamin Sapp, Balakrishnan Varadarajan, Yue Shen, Yi Shen, Yuning Chai, Cordelia Schmid and 2 more. Conference on Robot Learning, 2020

722 citations recorded by the source corpus.

Predicting the future behavior of moving agents is essential for real world applications. It is challenging as the intent of the agent and the corresponding behavior is unknown and intrinsically multimodal. Our key insight is that for prediction within a moderate time horizon, the future modes can be effectively captured by a set of target states. This leads to our target-driven trajectory prediction (TNT) framework. TNT has three stages which are trained end-to-end. It first predicts an agent's potential target states $T$ steps into the future, by encoding its interactions with the environment and the other agents. TNT then generates trajectory state sequences conditioned on targets. A final stage estimates trajectory likelihoods and a final compact set of trajectory predictions is selected. This is in contrast to previous work which models agent intents as latent variables, and relies on test-time sampling to generate diverse trajectories. We benchmark TNT on trajectory prediction of vehicles and pedestrians, where we outperform state-of-the-art on Argoverse Forecasting, INTERACTION, Stanford Drone and an in-house Pedestrian-at-Intersection dataset.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/c2cc3bac78dde326eddbc96951df3ab4fc56d3b6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting (2020, background): “• TNT [47] closely improves upon PECNet’s performance for K = 5 samples on SDD and is the prior state-of-the-art in that setting.”
Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion (2022, background): “Recently, the goals of pedestrians [28,29,52,53] are introduced in the trajectory prediction system as condition to analyze the probability of multiple plausible endpoints.”
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset (2021, background): “Motion forecasting has received increasing attention as a critical requirement for planning in autonomous driving systems [8, 14, 40, 36, 28, 34].”
DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets (2021, methodology): “The most notable algorithm is non-maximum suppression (NMS) [39], where only locally high-scored goals are selected.”
VectorMapNet: End-to-end Vectorized HD Map Learning (2022, methodology): “Beyond the pixel-level semantic map, our work extracts a consistent vectorized map around vehicles from surrounding cameras or LiDARs, which is friendly for downstream tasks like motion prediction [7, 30, 31].”
Query-Centric Trajectory Prediction (2023, methodology): “Anchor-based methods achieve this with the guidance of anchors, which facilitate multimodal prediction by leveraging predefined maneuvers [12], candidate trajectories [6, 39], or map-adaptive goals [53, 55].”
Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer (2022, background): “Note that while more advanced trajectory prediction methods [40, 41, 42] and safety controller [30, 43] can be used, we empirically found our dynamics propagation with moving average and linear programming controller sufficient.”
DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models (2024, background): “…30, 39, 45] is limited to detecting and tracking familiar objects, omitting rare objects and their unique attributes; motion prediction [16, 18, 33, 35, 56] and planning [2, 31, 38] focus on trajectory-level actions, often neglecting the decision-level interactions between objects and the vehicle.”
MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction (2021, background): “Other approaches model a discrete distribution over a set of trajectories (learned or ﬁxed a priori) [2, 14, 32, 54], or via a collection of trajectory samples drawn from a latent distribution and decoded by the model [1, 30, 33, 40, 41].”
Motion Transformer with Global Intention Localization and Local Movement Refinement (2022, methodology): “Goal-based methods [61, 40, 15, 30] are another kinds of models where they first estimate several goal points of the agents and then complete full trajectory for each goal.”
HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction (2022, methodology): “It is also used as the backbone network by some following works [22,32,52].”
Wayformer: Motion Forecasting via Simple & Efficient Attention Networks (2022, methodology): “Modeling relationships between entities is often presented as an information aggregation process, and models employ pooling [23, 34, 31, 35, 10, 28], soft-attention [10, 23] or graph neural networks [36, 33, 30].”
A Survey on Trajectory-Prediction Methods for Autonomous Driving (2022, methodology): “Using VectorNet to extract map features, Zhao et al. [139] propose a target-driven method called TNT, which deﬁnes sparse goal anchors and selects the best trajectory to the target, and DenseTNT [140] estimates dense goal candidates and get better results than TNT. Zeng et al. [141] use LaneRCNN to…”
GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation (2021, methodology): “All datasets provide the local HD-Map as a lanelet graph. b) Metrics: The metrics commonly used by both datasets are the MRk and minFDEk, which are respectively the Miss Rate and the minimum Final Displacement Error for the top k predictions, as well as the minimum Average Displacement Error minADEk”