Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction

By Abduallah A. Mohamed, Kun Qian, Mohamed Elhoseiny, Christian G. Claudel. Computer Vision and Pattern Recognition, 2020

1,028 citations recorded by the source corpus.

Better machine understanding of pedestrian behaviors enables faster progress in modeling interactions between agents such as autonomous vehicles and humans. Pedestrian trajectories are not only influenced by the pedestrian itself but also by interaction with surrounding objects. Previous methods modeled these interactions by using a variety of aggregation methods that integrate different learned pedestrians states. We propose the Social Spatio-Temporal Graph Convolutional Neural Network (Social-STGCNN), which substitutes the need of aggregation methods by modeling the interactions as a graph. Our results show an improvement over the state of art by 20% on the Final Displacement Error (FDE) and an improvement on the Average Displacement Error (ADE) with 8.5 times less parameters and up to 48 times faster inference speed than previously reported methods. In addition, our model is data efficient, and exceeds previous state of the art on the ADE metric with only 20% of the training data. We propose a kernel function to embed the social interactions between pedestrians within the adjacency matrix. Through qualitative analysis, we show that our model inherited social behaviors that can be expected between pedestrians trajectories. Code is available at https://github.com/abduallahmohamed/Social-STGCNN.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Safety, Risk, Reliability and Quality

View Paper: https://www.semanticscholar.org/paper/1b24169304c4e85f60beae5e198e61038cfb3095

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Safety, Risk, Reliability and Quality

Citations in the corpus

SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction (2021, methodology): “For example, the motion of a pedestrian is very easy to be disturbed by other pedestrians [11], close friends or colleagues are likely to walk in groups [32], and different pedestrians usually conduct similar social actions [38].”
Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos (2022, methodology): “Following the evaluation protocol in previous work [57, 59, 67, 89] that involves stochastic unit in trajectory estimation, we report the minimum of among 20 samples for trajectory evaluation.”
Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion (2022, methodology): “In addition, the spatialtemporal graph model is applied to jointly model the temporal clues and social interactions [15, 16, 30, 38, 44, 50].”
Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey (2023, methodology): “Model Specific Task Spatial learning Temporal Learning Fusion Architecture STGCN [18] Traffic state prediction Spectral GCN Gated TCN Factorized T-GCN [24] Traffic state prediction Spatial GCN GRU Factorized DCRNN [26] Traffic state prediction Spatial GCN GRU Coupled GMAN [118] Traffic state predict”
Query-Centric Trajectory Prediction (2023, methodology): “With the use of permutation-invariant set operators such as pooling [3, 12, 14, 20, 46], graph convolution [11, 31, 36, 53], and attention mechanism [24, 26, 30, 32, 34, 52], vectorbased methods can efficiently aggregate sparse information in traffic scenes.”
Core Challenges of Social Robot Navigation: A Survey (2021, background): “Recently, several deep learning-based approaches have been proposed for crowd prediction where an agent’s planning objective is not well defined (Alahi et al. 2016; Gupta et al. 2018; Vemula et al. 2018; Mohamed et al. 2020; Zhao and Oh 2021).”
HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction (2022, background): “Since social interactions are ubiquitous in traffic scenarios and significantly impact the future motions of traffic agents, many motion prediction approaches have considered the dependencies between agents’ behaviors and reasoned agent-agent interactions using social pooling [2,13,23], graph neural”