Peeking Into the Future: Predicting Future Person Activities and Locations in Videos

By Junwei Liang, Lu Jiang, Juan Carlos Niebles, Alexander G. Hauptmann, Li Fei-Fei. Computer Vision and Pattern Recognition, 2019

418 citations recorded by the source corpus.

Deciphering human behaviors to predict their future paths/trajectories and what they would do from videos is important in many applications. Motivated by this idea, this paper studies predicting a pedestrian's future path jointly with future activities. We propose an end-to-end, multi-task learning system utilizing rich visual features about human behavioral information and interaction with their surroundings. To facilitate the training, the network is learned with an auxiliary task of predicting future location in which the activity will happen. Experimental results demonstrate our state-of-the-art performance over two public benchmarks on future trajectory prediction. Moreover, our method is able to produce meaningful future activity prediction in addition to the path. The result provides the first empirical evidence that joint modeling of paths and activities benefits future path prediction.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/4a735d968cd6bd16e34c1aee7a0ce838782360b1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction (2020, methodology): “PIF [14] 360.”
Core Challenges of Social Robot Navigation: A Survey (2021, background): “Although beyond the scope of our survey, we mention that similar trends can be found in the autonomous driving literature, where behavior prediction is often conditioned on agents’ goal estimates [42, 134, 155, 156, 206, 280, 286] or multiagent interaction modes [210, 239].”
SPACE: A Simulator for Physical Interactions and Causal Learning in 3D Environments (2021, background): “The notion of drawing inspiration from conventional concepts in neuroscience and psychology to understand human perception of physical dynamics and causal reasoning has been around for some time [1, 5, 16, 19].”
SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction (2021, methodology): “We compare our method with nine state-of-the-art methods, including Vanilla LSTM [1], Social-LSTM [1], SGAN [11], Sophie [37], PITF [26], Social-BiGAT [23], Social-STGCNN [32], RSGB [38], and STAR [47], in the past four years.”