Social LSTM: Human Trajectory Prediction in Crowded Spaces

By Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robicquet, Li Fei-Fei, Silvio Savarese. 2016

3,459 citations recorded by the source corpus.

Subfields: Automotive Engineering, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

VAIN: Attentional Multi-agent Predictive Modeling (2017, neutral): “In Social LSTM [10], this was remedied by pooling a local neighborhood of interactions.”
From Recognition to Cognition: Visual Commonsense Reasoning (2018, neutral): “Commonsense Reasoning Our task unifies work involving reasoning about commonsense phenomena, such as physics [54, 84], social interactions [2, 77, 12, 27], procedure understanding [91, 3] and predicting what might happen next in a video [74, 17, 92, 78, 18, 64, 85].”
Dense-Captioning Events in Videos (2017, methodology): “Our proposed architecture (Figure 2) draws on architectural elements present in recent work on action proposal [8] and social human tracking [1] to tackle both these challenges.”
A Compositional Object-Based Approach to Learning Physical Dynamics (2016, background): “The NP is also a Markovian variant of the Social LSTM (Alahi et al., 2016); it sums the encodings of context objects after encoding each object independently, similar to the Social LSTM’s “social pooling.””
Learning to Decompose and Disentangle Representations for Video Prediction (2018, background): “Visual prediction is a core problem in computer vision that has been studied in several contexts, including activity prediction and early recognition [20, 30], human pose and trajectory forecasting [1, 18], and future frame prediction [22, 31, 39, 44].”
Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors (2017, background): “While Alahi [4] was able to predict human trajectories of crowds by modelling each human with an LSTM and jointly predicting the paths.”
r2p2: A ReparameteRized Pushforward Policy for Diverse, Precise Generative Path Forecasting (2018, methodology): “We include two commonly used sample metrics [3,8,15,25,37], despite the shortcomings illustrated in Fig.”
What will Happen Next? Forecasting Player Moves in Sports Videos (2017, background): “(F3, F4) (F5, F6) F[1,2]: (x,y) of player with ball F[3,4]: (x,y) of player F[5,6]: (x,y) of nearest defender”
Lidar for Autonomous Driving: The Principles, Challenges, and Trends for Automotive Lidar and Perception Systems (2020, background): “The authors in [41] proposed an encoder–decoder LSTM model to recognize maneuvers and predict trajectory.”
VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation (2020, methodology): “The adjacency matrix A can be provided a heuristic, such as using the spatial distances [2] between the nodes.”
The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs (2018, methodology): “As in prior works [1, 16, 49], we use a leave-one-out approach, training on 4 sets and testing on the remaining set.”
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting (2021, methodology): “One popular type of methods [25, 1, 15] first uses temporal models to summarize trajectory features over time for each agent independently and then feeds the temporal features to social models to obtain socially-aware agent features.”
Naturalistic Driver Intention and Path Prediction Using Recurrent Neural Networks (2018, background): “[25], which produces a path proposal with uncertainty.”
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks (2018, methodology): “Also, unlike S-LSTM which requires computing a occupancy grid for each pedestrian our pooling mechanism is a simple MLP followed by max pooling.”
Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction (2020, methodology): “sks, e.g., speech recognition [46,39], robot localization [14,36], robot decision making [23,37], and etc. RNNs have been also successfully applied to model the temporal motion pattern of pedestrians [1,16,21,53,22]. RNNs-based predictors make predictions with a Seq2Seq structure [41]. Additional st”
Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction (2020, methodology): “Based on the assumption that pedestrian trajectories follow multi-modal distributions, Social-GAN [6] extends Social LSTM [1] into a Recurrent Neural Network (RNN) based generative model.”
Deep Sequential Context Networks for Action Prediction (2017, background): “The prediction of future events was also investigated in other applications, such as predicting events in recommendation systems [16, 23], predicting future visual representation [29], and reasoning about the preferred path for a person [11, 1].”
Query-Centric Trajectory Prediction (2023, methodology): “With the use of permutation-invariant set operators such as pooling [3, 12, 14, 20, 46], graph convolution [11, 31, 36, 53], and attention mechanism [24, 26, 30, 32, 34, 52], vectorbased methods can efficiently aggregate sparse information in traffic scenes.”