Who are you with and where are you going?

By Kota Yamaguchi, Alexander C. Berg, Luis E. Ortiz, Tamara L. Berg. Computer Vision and Pattern Recognition, 2011

546 citations recorded by the source corpus.

We propose an agent-based behavioral model of pedestrians to improve tracking performance in realistic scenarios. In this model, we view pedestrians as decision-making agents who consider a plethora of personal, social, and environmental factors to decide where to go next. We formulate prediction of pedestrian behavior as an energy minimization on this model. Two of our main contributions are simple, yet effective estimates of pedestrian destination and social relationships (groups). Our final contribution is to incorporate these hidden properties into an energy formulation that results in accurate behavioral prediction. We evaluate both our estimates of destination and grouping, as well as our accuracy at prediction and tracking against state of the art behavioral model and show improvements, especially in the challenging observational situation of infrequent appearance observations–something that might occur in thousands of webcams available on the Internet.

Subfields: Computer Vision and Pattern Recognition, Ocean Engineering, Transportation

View Paper: https://www.semanticscholar.org/paper/af7657bbe96de2196869425b8f8c346e0ec50861

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Ocean Engineering

In Transportation

Citations in the corpus

Multi-Agent Tensor Fusion for Contextual Trajectory Prediction (2019, methodology): “Traditional methods for predicting or classifying trajectories model various kinds of interactions and constraints by hand-crafted features or cost functions [3, 5, 6, 8, 15, 22, 32].”
GSM: Graph Similarity Model for Multi-Object Tracking (2020, background): “Many early works extracted hand-craft appearance feature such as raw pixel template [ Yamaguchi et al. , 2011; Pellegrini et al. , 2009 ] , color histogram [ Izadinia et al. , 2012; Xiang et al. , 2015 ] , HOG [ Izadinia et al. , 2012; Kuo et al. , 2010 ] and so on.”
SR-LSTM: State Refinement for LSTM Towards Pedestrian Trajectory Prediction (2019, methodology): “Succeeding methods take more potential factors into account, such as pedestrian attributes [48, 52], walking group [30, 45], stationary group [48, 49].”
Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes (2016, methodology): “For instance, two leading families of methods for target trajectory forecasting (Social Forces [14,15,1,2] and Gaussian Processes [16,17,12]) have shown promising results on existing datasets [18,11]; however, they have never been tested at scale and in real-world scenarios where multiple classes of”
TrackFormer: Multi-Object Tracking with Transformers (2021, methodology): “Motion can be modelled for trajectory prediction [1, 25, 42] using a constant velocity assumption (CVA) [2, 9] or the social force model [25, 34, 43, 58].”
Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection (2017, methodology): “When predicting human behaviour the most common motion models are social force models (Helbing and Moln´ar, 1995; Koppula and Saxena, 2013; Pellegrini et al., 2010; Yamaguchi et al., 2011; Xu et al., 2012) which generate attractive and repulsive forces between pedestrians.”
Social LSTM: Human Trajectory Prediction in Crowded Spaces (2016, methodology): “We use the implementation of the Social Force model from [73] where several factors such as group affinity and predicted destinations have been modeled.”
3D Traffic Scene Understanding From Movable Platforms (2014, unknown): “Conclusions are drawn in Section 6.”
Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories With GANs (2019, methodology): “Optimization-based methods [23] optimize on-the-fly the parameters of an objective function hand-designed to cover relevant aspects of the motion.”
Temporal Perception and Prediction in Ego-Centric Video (2015, methodology): “More generally, some methods have looked at ideas related to human intent and goal inference for prediction using Stochastic Context Sensitive Grammars [28] or social models of behavior [39].”
Forecasting Interactive Dynamics of Pedestrians with Fictitious Play (2016, unknown): “It is also interesting to note that our feature naturally supports group behavior analysis, even though we do not explicitly model it as in [45].”
D Traffic Scene Understanding from Movable Platforms (2013, background): “For a long time, dynamic objects have been considered either in isolation [5], [26], [27], [52] or using simple motion [9], [23], [42], [47], [63] or social interaction [15], [62] models.”
Deep Affinity Network for Multiple Object Tracking (2018, background): “The appearance models focus on computing easy-to-track object features that encode appearances of local regions of objects or their bounding boxes [20], [21], [22].”
PIE: A Large-Scale Dataset and Models for Pedestrian Intention Estimation and Trajectory Prediction (2019, background): “Trajectory-based algorithms rely on observation of pedestrians’ past motion history and predict the location of the pedestrians in the future using contextual information such as 3D depth [14, 34, 39], social interactions [25, 42, 1, 31, 41], the ego-vehicle dynamics and the scene structure [16, 17,”
Multiple object tracking: A literature review (2014, methodology): “Employed Forces [2] repulsion, constancy, fidelity [80] repulsion, constancy, attraction, coherence [58] coherence”