Learning Lane Graph Representations for Motion Forecasting

By Ming Liang, Binh Yang, Rui Hu, Yun Chen, Renjie Liao, Song Feng, Raquel Urtasun. European Conference on Computer Vision, 2020

809 citations recorded by the source corpus.

We propose a motion forecasting model that exploits a novel structured map representation as well as actor-map interactions. Instead of encoding vectorized maps as raster images, we construct a lane graph from raw map data to explicitly preserve the map structure. To capture the complex topology and long range dependencies of the lane graph, we propose LaneGCN which extends graph convolutions with multiple adjacency matrices and along-lane dilation. To capture the complex interactions between actors and maps, we exploit a fusion network consisting of four types of interactions, actor-to-lane, lane-to-lane, lane-to-actor and actor-to-actor. Powered by LaneGCN and actor-map interactions, our model is able to predict accurate and realistic multi-modal trajectories. Our approach significantly outperforms the state-of-the-art on the large scale Argoverse motion forecasting benchmark.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/5c319a012777e6a4bc813645117ca1d260361bd6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting (2023, methodology): “A transition to attention-based methods [24, 33, 32] has led to a variety of new vector-based representations for map and trajectory data [13, 27].”
DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets (2021, methodology): “Sparse encoding methods [11, 20] (also called vectorized methods) were proposed recently.”
Graph Neural Networks for Intelligent Transportation Systems: A Survey (2023, background): “Baghbani et al. [109] explain that previous machine learning approaches that learn semantic representations from HD maps have two shortcomings: their rasterization process results in more or less information loss, and using a 2D convolution might be insufficient for capturing the complex topologies…”
GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation (2021, methodology): “The lanelet 96 features F are then updated through a GraphEncoder1 made of a sequence of four graph convolu97 tion operations similar to [13] in order to spread connectivity information: 98 F ← FW + ∑”
Multimodal Motion Prediction with Stacked Transformers (2021, methodology): “Graph neural networks [21] have emerged in response to problems that scenes cannot be easily represented by matrices of pixels or simple vectors.”
A Survey on Trajectory-Prediction Methods for Autonomous Driving (2022, methodology): “Liang et al. [138] use CNN to extract vehicle features and GCN to extract lane features from vector maps, and then combine these two features for trajectory prediction.”
VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning (2024, methodology): “Other approaches utilize vectorized representations and employ Graph Neural Networks [27] or Transformer models [13, 33, 36] for feature extraction and motion prediction.”
Motion Transformer with Global Intention Localization and Local Movement Refinement (2022, background): “LaneGCN [27] builds a lane graph toscalability capture map topology.”
Query-Centric Trajectory Prediction (2023, methodology): “Due to the lossy rendering, limited receptive field, and prohibitively high cost of raster-based methods, the research community has turned to a vector-based encoding scheme [14, 31, 50].”
Wayformer: Motion Forecasting via Simple & Efficient Attention Networks (2022, methodology): “A popular alternative is to use an entity-centric approach, where agent state history is typically encoded via sequence modeling techniques like RNNs [36], [27], [1], [41] or temporal convolutions [32].”
MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction (2021, methodology): “Cross attention has also been used to condition one input type (e.g. agent encodings) on another (e.g. road lanes) [20, 32, 35].”
VAD: Vectorized Scene Representation for Efficient Autonomous Driving (2023, neutral): “3.4).”
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset (2021, neutral): “[1, 3, 7, 5, 8, 12, 11, 14, 17, 20, 22, 25, 39].”