Tracking People by Learning Their Appearance

By Deva Ramanan, D.A. Forsyth, Andrew Zisserman. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007

445 citations recorded by the source corpus.

An open vision problem is to automatically track the articulations of people from a video sequence. This problem is difficult because one needs to determine both the number of people in each frame and estimate their configurations. But, finding people and localizing their limbs is hard because people can move fast and unpredictably, can appear in a variety of poses and clothes, and are often surrounded by limb-like clutter. We develop a completely automatic system that works in two stages; it first builds a model of appearance of each person in a video and then it tracks by detecting those models in each frame ("tracking by model-building and detection"). We develop two algorithms that build models; one bottom-up approach groups together candidate body parts found throughout a sequence. We also describe a top-down approach that automatically builds people-models by detecting convenient key poses within a sequence. We finally show that building a discriminative model of appearance is quite helpful since it exploits structure in a background (without background-subtraction). We demonstrate the resulting tracker on hundreds of thousands of frames of unscripted indoor and outdoor activity, a feature-length film ("Run Lola Run"), and legacy sports footage (from the 2002 World Series and 1998 Winter Olympics). Experiments suggest that our system 1) can count distinct individuals, 2) can identify and track them, 3) can recover when it loses track, for example, if individuals are occluded or briefly leave the view, 4) can identify body configuration accurately, and 5) is not dependent on particular models of human motion

View Paper: https://www.semanticscholar.org/paper/e539fd817b9f26de52a352fb2c3adc7dd2fb06b9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Event Detection in Crowded Videos (2007, background): “And while recent work has demonstrated significant progress in cluttered environments [22], tracking remains challenging in such environments, and the tracker output tends to be noisy.”
Where's Waldo: Matching people in images of crowds (2011, methodology): “Labeling pixels inside a part-specific mask as positives (yj = 1) and those outside as negatives (yj = −1) (with a 10-pixel band separating them), we learn a logistic regression classifier similar to [9] by computing wpart = argminw ∑ j log(1 + exp(−yjw T xj)).”
Tracking-Learning-Detection (2012, background): “Various representations of the object are used in practice, for example, points [2], [3], [4], articulated models [5], [6], [7], contours [8], [9], [10], [11], or optical flow [12], [13], [14].”
Articulated pose estimation with tiny synthetic videos (2015, background): “also well aligned with [26], [39] in that they attempt to use consistency existing throughout the video.”
A Review on Video-Based Human Activity Recognition (2013, background): “[53] also use appearance-based features for human tracking.”
People-tracking-by-detection and people-detection-by-tracking (2008, background): “[19] propose a two-stage approach that first builds a model of the appearance of individual people, and then tracks them by detecting those models in each frame.”
Multi-view 3D Human Pose Estimation in Complex Environment (2011, methodology): “Lastly, the system expects subjects to be roughly in a standing position for pose initialization, for computational cost reasons.”