Unsupervised Activity Perception in Crowded and Complicated Scenes Using Hierarchical Bayesian Models

By Xiaogang Wang, Xiaoxu Ma, W. Grimson. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008

650 citations recorded by the source corpus.

We propose a novel unsupervised learning framework to model activities and interactions in crowded and complicated scenes. Hierarchical Bayesian models are used to connect three elements in visual surveillance: low-level visual features, simple "atomic" activities, and interactions. Atomic activities are modeled as distributions over low-level visual features, and multi-agent interactions are modeled as distributions over atomic activities. These models are learnt in an unsupervised way. Given a long video sequence, moving pixels are clustered into different atomic activities and short video clips are clustered into different interactions. In this paper, we propose three hierarchical Bayesian models, Latent Dirichlet Allocation (LDA) mixture model, Hierarchical Dirichlet Process (HDP) mixture model, and Dual Hierarchical Dirichlet Processes (Dual-HDP) model. They advance existing language models, such as LDA [1] and HDP [2]. Our data sets are challenging video sequences from crowded traffic scenes and train station scenes with many kinds of activities co-occurring. Without tracking and human labeling effort, our framework completes many challenging visual surveillance tasks of board interest such as: (1) discovering typical atomic activities and interactions; (2) segmenting long video sequences into different interactions; (3) segmenting motions into different activities; (4) detecting abnormality; and (5) supporting high-level queries on activities and interactions.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/bdcaffe643160dde1fabd272487ea9c209cb70a9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Crowd Analysis Using Computer Vision Techniques (2010, background): “Wang et al. [43] proposed an unsupervised learning framework to model activities and interactions in crowded and complicated scenes.”
Unstructured human activity detection from RGBD images (2011, methodology): “We have also made the dataset and code available open-source at: http://pr.cs.cornell.edu/humanactivities”
Understanding collective crowd behaviors: Learning a Mixture model of Dynamic pedestrian-Agents (2012, methodology): “It has important applications, such as event recognition [12], traffic flow estimation [23], behavior prediction [2], and crowd simulation [20].”
Sparse reconstruction cost for abnormal event detection (2011, background): “There are also some trajectory-based approaches to locate objects by tracking or frame-difference, such as [10], [24], [21] and [13].”
D Traffic Scene Understanding from Movable Platforms (2013, background): “Methods for unsupervised activity recognition and abnormality detection [44], [59] are able to recover spatiotemporal dependencies using video sequences from a static camera mounted on top of a building.”
Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach (2011, background): “[20], [21], [22] have developed hierarchical Bayesian models to group co-occurring motions into actions and activities, jointly and without supervision.”
Intelligent multi-camera video surveillance: A review (2013, background): “Therefore, people prefer unsupervised approaches (Brand and Kettnaker , 2000; Song et al., 2003; Wang et al., 2006, 2009) which can automatically learn the models of activities without labeling training samples.”
On Human Action (2011, background): “analysis of the similarities between local frames [21] or by a more holistic approach that attempts to recover a compact description based upon co-occurrence of the lower level properties [32, 38, 48].”
Video anomaly detection based on local statistical aggregates (2012, background): “This insight helps collapse the large ambient data dimension for detecting local anomalies.”
Video-Based Human Behavior Understanding: A Survey (2013, background): “Knowledge of the scene can be used to train for a specific area [55], [56].”
3D Traffic Scene Understanding From Movable Platforms (2014, background): “Methods for unsupervised activity recognition and abnormality detection [44], [59] are able to recover spatio-temporal dependencies using video sequences from a static camera mounted on top of a building.”
A temporal Bayesian model for classifying, detecting and localizing activities in video sequences (2012, methodology): “For example, Topic Modeling or Latent Dirichlet Allocation (LDA) [2] has been used to cluster low level features from videos into activities [25, 27, 24, 10].”
Abnormal event detection in crowded scenes using sparse representation (2013, background): “Due to the object template can be initialized in the uncrowded scene, there are also some trajectory-based approaches by tracking the objects, such as [32,12,28,15].”
Topic models for scene analysis and abnormality detection (2009, methodology): “using just optical flow words as was done in [8].”
STREETS: A Novel Camera Network Dataset for Traffic Flow (2019, background): “There are many other traffic camera datasets that utilize fewer than ten cameras and limit analysis to one intersection at a time [13, 14, 15, 16, 17, 18, 19, 20].”