Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks

By Vineet Kosaraju, Amir Sadeghian, Roberto Mart'in-Mart'in, Ian David Reid, Seyed Hamid Rezatofighi, Silvio Savarese. Neural Information Processing Systems, 2019

738 citations recorded by the source corpus.

Predicting the future trajectories of multiple interacting agents in a scene has become an increasingly important problem for many different applications ranging from control of autonomous vehicles and social robots to security and surveillance. This problem is compounded by the presence of social interactions between humans and their physical interactions with the scene. While the existing literature has explored some of these cues, they mainly ignored the multimodal nature of each human's future trajectory. In this paper, we present Social-BiGAT, a graph-based generative adversarial network that generates realistic, multimodal trajectory predictions by better modelling the social interactions of pedestrians in a scene. Our method is based on a graph attention network (GAT) that learns reliable feature representations that encode the social interactions between humans in the scene, and a recurrent encoder-decoder architecture that is trained adversarially to predict, based on the features, the humans' paths. We explicitly account for the multimodal nature of the prediction problem by forming a reversible transformation between each scene and its latent noise vector, as in Bicycle-GAN. We show that our framework achieves state-of-the-art performance comparing it to several baselines on existing trajectory forecasting benchmarks.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/e154205a6f3178ae513a1120fe415181fa565640

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction (2021, methodology): “To model the interactions between pedestrians, extensive works [31, 2, 11, 23, 19, 32, 46] have been done in the past few years, in which the weighting-by-distance methods [31, 2, 11, 32] and the attention-based meth-ods [23, 19, 46, 8, 17, 18] have achieved the state-of-the-art results in…”
Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion (2022, methodology): “For the ETH/UCY dataset, we followed the leave one out cross-validation evaluation strategy such that we trained our model on four scenes and tested on the remaining one [12,15,19,38].”
Multi-Modal Fusion Transformer for End-to-End Autonomous Driving (2021, background): “Attention for Autonomous Driving: Attention has been explored in the context of driving for lane changing [13], object detection [11, 32] and motion forecasting [32, 50, 49, 28, 15, 30, 29, 56].”
How many Observations are Enough? Knowledge Distillation for Trajectory Forecasting (2022, neutral): “In this regard, the widely adopted 8 - 12 protocol [1,14,19, 21,33,47,51] ( i.e. , 8 input time steps and 12 ones for prediction), which require data collected at 2 .”
TransFuser: Imitation With Transformer-Based Sensor Fusion for Autonomous Driving (2022, background): “Attention has been explored in the context of driving for lane changing [65], object detection [31], [66], [67], [68], motion forecasting [31], [69], [70], [71], [72], [73], [74], [75], [76], [77], driver attention prediction [78], [79] and recently also for end-to-end driving [39], [41].”
Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction (2020, methodology): “One key difference between our model Social-STGCNN and Social-BiGAT is that we directly model pedestrian trajectories as a graph from the beginning, where we give meaningful values for vertices.”
Transformer Networks for Trajectory Forecasting (2020, methodology): “Notably, TF is better than the Social-LSTM [18], and it outperforms the Social Attention [27] in terms of FAD too, by a large margin.”
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting (2021, methodology): “…recent trajectory prediction methods start to model the trajectory distribution with deep generative models [21, 13, 40] such as conditional variational autoencoders (CVAEs) [27, 57, 19, 46, 51, 45], generative adversarial networks (GANs) [15, 44, 25, 62], and normalizing ﬂows (NFs) [41, 42, 14].”
Wayformer: Motion Forecasting via Simple & Efficient Attention Networks (2022, methodology): “The order of applying attention over {temporal, social/spatial} dimensions has been studied with two different common patterns: (a) Temporal first [31, 35, 45] (b) Social/Spatial first [46, 47].”
Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data (2020, methodology): “For the ETH and UCY datasets, a leave-one-out strategy is used for evaluation, similar to previous works [1,13,18,27,40], where the model is trained on four datasets and evaluated on the held-out fifth.”
How Attentive are Graph Attention Networks? (2021, background): “…spread to a variety of domains (Wang et al., 2019a; Qiu et al., 2018; Yang et al., 2020; Wang et al., 2019c; Huang and Carley, 2019; Ma et al., 2020; Kosaraju et al., 2019; Nathani et al., 2019; Wu et al., 2020; Zhang et al., 2020) and has become the default implementation of “graph attention…”
Human motion trajectory prediction: a survey (2019, background): “, 2019) or extracted features from a top-down image (Kosaraju et al., 2019; Sadeghian et al., 2019; Tang and Salakhutdinov, 2019; Xue et al., 2018) can be used to capture people preferences in walking on a particular type of surfaces.”