Social Attention: Modeling Attention in Human Crowds

By Anirudh Vemula, Katharina Muelling, Jean Oh. IEEE International Conference on Robotics and Automation, 2017

723 citations recorded by the source corpus.

Robots that navigate through human crowds need to be able to plan safe, efficient, and human predictable trajectories. This is a particularly challenging problem as it requires the robot to predict future human trajectories within a crowd where everyone implicitly cooperates with each other to avoid collisions. Previous approaches to human trajectory prediction have modeled the interactions between humans as a function of proximity. However, that is not necessarily true as some people in our immediate vicinity moving in the same direction might not be as important as other people that are further away, but that might collide with us in the future. In this work, we propose Social Attention, a novel trajectory prediction model that captures the relative importance of each person when navigating in the crowd, irrespective of their proximity. We demonstrate the performance of our method against a state-of-the-art approach on two publicly available crowd datasets and analyze the trained attention model to gain a better understanding of which surrounding agents humans attend to, when navigating in a crowd.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Ocean Engineering

View Paper: https://www.semanticscholar.org/paper/7c739dddc905c967e0b432dc7515cb1f4b82e580

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Ocean Engineering

Citations in the corpus

From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting (2020, background): “[42] proposes to use attention to model target agent’s interaction with other agent’s.”
Deep Local Trajectory Replanning and Control for Robot Navigation (2019, methodology): “puting obstacle-free routes in static environments, and we use deep networks because they have a great capacity for reasoning about raw sensor data and modeling complex phenomena like social behavior [48, 49]. Note that the learning component of our hierarchical navigation approach has more structur”
Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories With GANs (2019, methodology): “In [20], groups of agents are modeled as spatio-temporal graphs where edges (temporal and spatial) are associated to RNNs.”
Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion (2022, methodology): “Some methods apply the attention models [10, 19, 37, 47, 51] to explore the key interactions of the crowd.”
STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction (2019, methodology): “[33] proposed a novel spatial-temporal graph, which uses an attention module to merge information from different edgeRNNs.”
Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data (2020, methodology): “As a result, they are commonly found as a core component of human trajectory models [1,21,46].”
Core Challenges of Social Robot Navigation: A Survey (2021, background): “Recently, several deep learning-based approaches have been proposed for crowd prediction where an agent’s planning objective is not well defined (Alahi et al. 2016; Gupta et al. 2018; Vemula et al. 2018; Mohamed et al. 2020; Zhao and Oh 2021).”
TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents (2018, methodology): “The edge between two instance nodes in one frame is called spatial edge (Jain et al. 2016; Vemula et al. 2017), which can transfer the interaction information between two trafﬁc-agents in spatial space.”
The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs (2018, methodology): “, graphs directly define the structure of a deep learning architecture) [24, 25, 31, 49] and methods which take a graph as input and provide n-step predictions as their output [5, 6, 27, 42].”
AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting (2021, methodology): “Prior work has also used RNNs to model the temporal dynamics of 3D human pose [11, 58, 60].”
Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Reinforcement Learning (2018, methodology): “other works model the social interactions through spatiotemporal graphs, where an attention model is introduced recently to learn the relative importance of each agent [14].”
How many Observations are Enough? Knowledge Distillation for Trajectory Forecasting (2022, neutral): “Several solutions [6, 19, 21, 37, 39, 43] consider agents as graph nodes whose features are represented by their hidden states.”
Multi-Agent Tensor Fusion for Contextual Trajectory Prediction (2019, background): “…spatial interactions by sharing parameters across agents and space, while recent approaches like Social Pooling [1, 13] or Attention mechanisms [31] cannot explicitly reason about spatial relationships among agents and cannot reason about these relationships at multiple spatial scales.”
Transformer Networks for Trajectory Forecasting (2020, background): “Social models and context: Enabled by the flexibility of the LSTM machinery, best performance has been recently achieved by modelling the social interaction [4], [5], [25] among people and the scene context [9], [7], [6], aided by tracking dynamics [26] and the spatio-temporal relations among neighb”
SR-LSTM: State Refinement for LSTM Towards Pedestrian Trajectory Prediction (2019, methodology): “[39] compute a soft attention score from the hidden states of the designed edgeRNNs, which gives an importance value for each neighbor.”
Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review (2020, background): “vehicles, pedestrian, and cyclist) are used with different weights, obtained by soft attention mechanism [33], corresponding to their impacts on TV’s behaviour.”
SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction (2021, methodology): “TPHT [30] represents each pedestrian by an LSTM and employs a soft-attention mechanism [42] to model interactions between pedestrians.”
Human motion trajectory prediction: a survey (2019, methodology): “Several others can be extended to include further types of contextual information (e.g. Alahi et al. (2016); Trautman and Krause (2010); Vemula et al. (2018); Pfeiffer et al. (2018); Bartoli et al. (2018)) but such extension may lead to involved learning, data efﬁciency and generalization issues…”
TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions (2018, methodology): “There is considerable work on trajectory prediction for moving agents [3], [4], [5], [6], [7], [8], [6], [7].”