Learning Patterns of Activity Using Real-Time Tracking

By Chris Stauffer, W. Grimson. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000

3,734 citations recorded by the source corpus.

Our goal is to develop a visual monitoring system that passively observes moving objects in a site and learns patterns of activity from those observations. For extended sites, the system will require multiple cameras. Thus, key elements of the system are motion tracking, camera coordination, activity classification, and event detection. In this paper, we focus on motion tracking and show how one can use observed motion to learn patterns of activity in a site. Motion segmentation is based on an adaptive background subtraction method that models each pixel as a mixture of Gaussians and uses an online approximation to update the model. The Gaussian distributions are then evaluated to determine which are most likely to result from a background process. This yields a stable, real-time outdoor tracker that reliably deals with lighting changes, repetitive motions from clutter, and long-term scene changes. While a tracking system is unaware of the identity of any object it tracks, the identity remains the same for the entire tracking sequence. Our system leverages this information by accumulating joint co-occurrences of the representations within a sequence. These joint co-occurrence statistics are then used to create a hierarchical binary-tree classification of the representations. This method is useful for classifying sequences, as well as individual instances of activities in a site.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1bfe26fac93ad96c81cf1a580b9e7744477f56aa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey (2008, background): “For example, some authors [86,108, 109, 114] propose to select the rule for each pixel following it previous classification and other ones [71] propose to select the rule for all the pixels following the illumination change detected in the scene.”
Detecting unusual activity in video (2004, background): “This over-dependence on the feature set has been a general weakness for most unsupervised approaches [13].”
Learning activity patterns using fuzzy self-organizing neural network (2004, methodology): “Johnson et al. [24] generalize this method to the learning of interactions among humans, for example shaking hands.”
Robust Real-Time Unusual Event Detection using Multiple Fixed-Location Monitors (2008, result): “In contrast with most previous works reported, our algorithm satisfies all of these requirements (having of course its own limitations which will be discussed).”
Semi-supervised adapted HMMs for unusual event detection (2005, methodology): “…that are relevant, but whose occurrence rate is either expected to be very low or cannot be anticipated at all, constitutes a problem which has recently attracted attention in computer vision and multimodal processing under an umbrella of names (abnormal, unusual, or rare events) [17, 19, 6].”
Learning Motion Patterns of People for Compliant Robot Motion (2005, background): “The goal of the work by Stauffer and Grimson (2000) is also to detect unusual events.”
Discriminative Latent Models for Recognizing Contextual Group Activities (2012, methodology): “First, we perform background subtraction using the OpenCV implementation of the standard Gaussian Mixture Model (GMM) [46] to obtain the foreground regions.”
ViBE: A powerful random technique to estimate the background in video sequences (2009, methodology): “Index Terms— Surveillance, Pattern recognition, Signal analysis, Video signal processing”
Tracking multiple humans in complex situations (2004, methodology): “The background model can be easily replaced with a more complex one (e.g., one with a multi- Guassian model [40] or one which can start with moving objects in the scene [15]) if needed.”
Beyond Actions: Discriminative Models for Contextual Group Activities (2010, neutral): “First, we perform background subtraction using the OpenCV implementation of the standard Gaussian Mixture Model (GMM) [38] to obtain the foreground regions.”
Ensemble Tracking (2005, background): “Visual tracking is a critical step in many machine vision applications such as surveillance [15], driver assistance systems [1] or human-computer interactions [3].”
Recognition of group activities using dynamic probabilistic networks (2003, background): “Probabilistic graph models including both temporal sequential models such as Hidden Markov Models (HMMs) and static causal models such as Bayesian Belief Networks (BBNs) have received enormous attention in recent years for modelling and recognising visual behaviours of activities captured in video, ”
Estimating the number of people in crowded scenes by MID based foreground segmentation and head-shoulder detection (2008, methodology): “It is difﬁcult to segment foreground by background modeling methods, like GMM (Gaussian Mixture Model) [9], in places with people gathering and waiting.”
Algorithms for cooperative multisensor surveillance (2001, background): “Conventional approaches to moving object detection include temporal differencing [35], [36]; background subtraction [6], [34], [37], [38] and optical flow [39]–[41].”
View-invariance in action recognition (2001, neutral): “Or they can be high level descriptions like “opening a door”, “starting a car”, “throwing a ball”, or more abstractly “pick up”, “put down”, “push”, “pull”, “drop”, “throw”, etc. Motion verbs can also be associated with motion events.”
Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame (2000, background): “In a related paper [9], we focus on classifying activities as recorded by a distributed set of sensors by considering patterns of activity in a common global coordinate frame.”
Trajectory-Based Anomalous Event Detection (2008, methodology): “The acquired video data consist in a 51-min-long sequence at 320 240 resolution, during which 1430 vehicles were detected by means of multi-Gaussian background-subtraction techniques as in [ 19 ] and tracked using a combination of Kalman-based and CamShift trackers.”
Detecting Moving Objects, Ghosts, and Shadows in Video Streams (2003, methodology): “In the absence of any a priori knowledge about target and environment, the most widely adopted approach for moving object detection with fixed camera is based on background subtraction [1], [2], [3], [4], [5], [6], [7], [8], [9].”