CoverNet: Multimodal Behavior Prediction Using Trajectory Sets

By Tung Phan-Minh, Elena Corina Grigore, Freddy A. Boulton, Oscar Beijbom, Eric M. Wolff. Computer Vision and Pattern Recognition, 2019

495 citations recorded by the source corpus.

We present CoverNet, a new method for multimodal, probabilistic trajectory prediction for urban driving. Previous work has employed a variety of methods, including multimodal regression, occupancy maps, and 1-step stochastic policies. We instead frame the trajectory prediction problem as classification over a diverse set of trajectories. The size of this set remains manageable due to the limited number of distinct actions that can be taken over a reasonable prediction horizon. We structure the trajectory set to a) ensure a desired level of coverage of the state space, and b) eliminate physically impossible trajectories. By dynamically generating trajectory sets based on the agent's current state, we can further improve our method's efficiency. We demonstrate our approach on public, real world self-driving datasets, and show that it outperforms state-of-the-art methods.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Transportation

View Paper: https://www.semanticscholar.org/paper/e73a1061e36d6e55ee4d5cf43d0159e26ab59a60

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Transportation

Citations in the corpus

DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets (2021, background): “Some approaches aimed to address mode collapse [25, 36, 23, 10, 3].”
Motion Transformer with Global Intention Localization and Local Movement Refinement (2022, background): “Some other works [9, 21, 33, 37, 41] parameterize multi-modal predictions with Gaussian Mixture Models (GMMs) to generate compact distribution.”
FIERY: Future Instance Prediction in Bird’s-Eye View from Surround Monocular Cameras (2021, background): “Other work [8, 46, 36] has been done on probabilistic multihypothesis trajectory prediction, however all assume access to top-down rasterised representations as inputs.”
Recent Advancements in End-to-End Autonomous Driving Using Deep Learning: A Survey (2023, background): “CoverNet [38] Formulation of classification problem over the set of diverse trajectories CNN CNN CNN Trajectory set generator”
MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction (2021, methodology): “Agent trajectory decoding choices are similar to choices for encoding motion history, with the exception of methods that do lookup on a fixed or learned trajectory database [2, 37].”
GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation (2021, methodology): “Epistemic uncertainty outlines some knowledge that can’t 21 be known by the observer at prediction time: what is the car destination, will it choose to overtake 22 the car by the left or stay behind ? Recent methods use multimodal outputs based on anchors [3, 4], 23 predefined learning heads [5] or ”
A Survey on Trajectory-Prediction Methods for Autonomous Driving (2022, background): “In [92], a set of possible future trajectories is generated by the vehicle state (velocity, acceleration, and yaw rate) and the raster image, the trajectory with the highest probability is found as the future trajectory through analyzing semantic features.”
VAD: Vectorized Scene Representation for Efficient Autonomous Driving (2023, unknown): “3.4).”
Multimodal Motion Prediction with Stacked Transformers (2021, background): “Recent motion prediction methods mainly follow probabilistic approaches [19, 16, 28] or proposal-based approaches [33, 26, 5, 11] to address the aforementioned issue.”
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset (2021, unknown): “[1, 3, 7, 5, 8, 12, 11, 14, 17, 20, 22, 25, 39].”
VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning (2024, methodology): “In terms of scene representation, some works adopt ras-terized image representations and employ CNN networks for prediction [3, 37].”
TNT: Target-driveN Trajectory Prediction (2020, methodology): “First, future time steps are conditionally independent, which makes our model computationally efficient by avoiding sequential predictions, as is done in [33, 34, 21, 31].”
Query-Centric Trajectory Prediction (2023, methodology): “Anchor-based methods achieve this with the guidance of anchors, which facilitate multimodal prediction by leveraging predefined maneuvers [12], candidate trajectories [6, 39], or map-adaptive goals [53, 55].”
MP3: A Unified Model to Map, Perceive, Predict and Plan (2021, background): “This approach provides a large set of trajectories from expert demonstrations while avoiding random sampling or arbitrary choices of acceleration/steering profiles [36, 53].”