Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks

By Agrim Gupta, Justin Johnson, Li Fei-Fei, Silvio Savarese, Alexandre Alahi. 2018

2,420 citations recorded by the source corpus.

Understanding human motion behavior is critical for autonomous moving platforms (like self-driving cars and social robots) if they are to navigate human-centric environments. This is challenging because human motion is inherently multimodal: given a history of human motion paths, there are many socially plausible ways that people could move in the future. We tackle this problem by combining tools from sequence prediction and generative adversarial networks: a recurrent sequence-to-sequence model observes motion histories and predicts future behavior, using a novel pooling mechanism to aggregate information across people. We predict socially plausible futures by training adversarially against a recurrent discriminator, and encourage diverse predictions with a novel variety loss. Through experiments on several datasets we demonstrate that our approach outperforms prior work in terms of accuracy, variety, collision avoidance, and computational complexity.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

Stochastic Prediction of Multi-Agent Interactions from Partial Observations (2019, methodology): “Social-RNN : one RNN per agent, agents interact via the pooling mechanism in Social GAN (Gupta et al., 2018).”
Deep Imitative Models for Flexible Inference, Planning, and Control (2018, background): “Typically this involves recurrent neural networks combined with Gaussian density layers or generative models based on some context inputs such as LIDAR, images, or known positions of external agents (Lee et al., 2017; Schmerling et al., 2018; Zyner et al., 2018; Gupta et al., 2018; Ma et al., 2017).”
A Survey on Generative Adversarial Networks: Variants, Applications, and Training (2020, background): “ng [178, 179], spatial representation learning [180], mobile user profiling [181], data augmentation [182, 183], heterogeneous information networks [184], privacy-preserving [185 - 188], social robot [189], cipher cracking [190], auxiliary automatic driving [236], continual learning [237], molecule ”
Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks (2018, background): “Authors of [28], [29] applied LSTM to predict pedestrians’ future trajectories with social interactions.”
SR-LSTM: State Refinement for LSTM Towards Pedestrian Trajectory Prediction (2019, methodology): “We follow the leave-one-out evaluation methodology in [11].”
DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets (2021, background): “Other approaches generate a set of trajectories but only perform regression on the closest one during training [12, 20, 8], namely using variety loss.”
SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction (2021, methodology): “For example, the motion of a pedestrian is very easy to be disturbed by other pedestrians [11], close friends or colleagues are likely to walk in groups [32], and different pedestrians usually conduct similar social actions [38].”
MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Prediction (2021, background): “SceneTransformer [35] polyline attention attention attention states Weighted set GOHOME [21] GNN 1D conv + GRU GNN MLP states heatmap MP3 [11] raster raster conv conv cost function Weighted samples CoverNet [37] raster raster conv lookup states GMM w/ dynamic anchors DESIRE [30] raster GRU spatial p”
Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Reinforcement Learning (2018, methodology): “Inspired by [13]–[15], our model extracts features for pairwise interactions between the robot and each human and captures the interactions among humans via local maps.”
Core Challenges of Social Robot Navigation: A Survey (2021, background): “Note that human motion prediction is a relevant and active field that has received much attention recently [2, 90, 178, 214, 270, 287].”
Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos (2019, background): “Recently, more effort has been invested into unsupervised learning of human motion in social settings [1, 12, 23] and single pose configuration [10].”
TNT: Target-driveN Trajectory Prediction (2020, methodology): “For example, DESIRE [1] used a conditional VAE [26] while PRECOG [3] used flow-based generative models [27]; SocialGAN [2] proposed an adversarial discriminator to predict realistic futures; Hong et al. [22] modeled the motion patterns with a latent Gaussian mixture model.”
Human Action Recognition and Prediction: A Survey (2018, methodology): “RNN/LSTM networks [6,119,195,57], which have the capability of generating long sequences.”
One Thousand and One Hours: Self-driving Motion Prediction Dataset (2020, background): “Deep learning solutions leveraging the birds-eye-view (BEV) representation of the scene [1, 10, 15, 7, 5, 12] or graph neural networks [4, 8] have established themselves as the leading solutions for this task.”
VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation (2020, background): “Beyond the autonomous driving domain, there is more general interest to predict the intents of interacting agents, such as for pedestrians [2, 13, 24], human activities [28] or for sports players [12, 26, 32, 33].”
A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications (2020, methodology): “GANs have been widely used in many other areas such as malware detection [434], chess game playing [435], steganography [436]–[439], privacy-preserving [440]–[442], social robot [443], and network pruning [444], [445].”
Multi-Agent Tensor Fusion for Contextual Trajectory Prediction (2019, methodology): “We adopt exactly the same experimental settings, splits and error measures as [13].”