SR-LSTM: State Refinement for LSTM Towards Pedestrian Trajectory Prediction

By Pu Zhang, Wanli Ouyang, Pengfei Zhang, Jianru Xue, Nanning Zheng. Computer Vision and Pattern Recognition, 2019

548 citations recorded by the source corpus.

In crowd scenarios, reliable trajectory prediction of pedestrians requires insightful understanding of their social behaviors. These behaviors have been well investigated by plenty of studies, while it is hard to be fully expressed by hand-craft rules. Recent studies based on LSTM networks have shown great ability to learn social behaviors. However, many of these methods rely on previous neighboring hidden states but ignore the important current intention of the neighbors. In order to address this issue, we propose a data-driven state refinement module for LSTM network (SR-LSTM), which activates the utilization of the current intention of neighbors, and jointly and iteratively refines the current states of all participants in the crowd through a message passing mechanism. To effectively extract the social effect of neighbors, we further introduce a social-aware information selection mechanism consisting of an element-wise motion gate and a pedestrian-wise attention to select useful message from neighboring pedestrians. Experimental results on two public datasets, i.e. ETH and UCY, demonstrate the effectiveness of our proposed SR-LSTM and we achieve state-of-the-art results.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/7623616066783e73ad29e84885e1b56dbcf39e97

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Transformer Networks for Trajectory Forecasting (2020, background): “Recent works brought up some issues with the ETH+UCY dataset, [12] showed that Hotel contains trajectory that go in a different direction than most of the ones in the other 4 dataset, so learning an environmental prior can be difficult without data augmentation like rotation; [30]”
Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction (2020, background): “SR-LSTM-2 [30] 64.”
Human motion trajectory prediction: a survey (2019, methodology): “Several learning-based approaches that use LSTMs (Alahi et al. 2016; Bartoli et al. 2018; Varshneya and Srinivasaraghavan 2017; Pfeiffer et al. 2018; Zhang et al. 2019; Shi et al. 2019) may be capable of implicitly learning intra-and inter-group coherence behavior, however only the work by Bisagno e”
Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion (2022, methodology): “Some methods apply the attention models [10, 19, 37, 47, 51] to explore the key interactions of the crowd.”
Core Challenges of Social Robot Navigation: A Survey (2021, background): “Although beyond the scope of our survey, we mention that similar trends can be found in the autonomous driving literature, where behavior prediction is often conditioned on agents’ goal estimates [42, 134, 155, 156, 206, 280, 286] or multiagent interaction modes [210, 239].”
Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction (2020, methodology): “Crowd trajectory prediction is of fundamental importance to both the computer vision [1,16,53,21,22] and robotics [34,33] community.”
How many Observations are Enough? Knowledge Distillation for Trajectory Forecasting (2022, methodology): “To overcome this issue, we follow [13, 14] and employ data augmentation by applying a random rotation to each position inside each mini-batch.”