Articulated Human Detection with Flexible Mixtures of Parts

By Yi Yang, Deva Ramanan. 2013

933 citations recorded by the source corpus.

We describe a method for articulated human detection and human pose estimation in static images based on a new representation of deformable part models. Rather than modeling articulation using a family of warped (rotated and foreshortened) templates, we use a mixture of small, nonoriented parts. We describe a general, flexible mixture model that jointly captures spatial relations between part locations and co-occurrence relations between part mixtures, augmenting standard pictorial structure models that encode just spatial relations. Our models have several notable properties: 1) They efficiently model articulation by sharing computation across similar warps, 2) they efficiently model an exponentially large set of global mixtures through composition of local mixtures, and 3) they capture the dependency of global geometry on local appearance (parts look different at different locations). When relations are tree structured, our models can be efficiently optimized with dynamic programming. We learn all parameters, including local appearances, spatial relations, and co-occurrence relations (which encode local rigidity) with a structured SVM solver. Because our model is efficient enough to be used as a detector that searches over scales and image locations, we introduce novel criteria for evaluating pose estimation and human detection, both separately and jointly. We show that currently used evaluation criteria may conflate these two issues. Most previous approaches model limbs with rigid and articulated templates that are trained independently of each other, while we present an extensive diagnostic evaluation that suggests that flexible structure and joint training are crucial for strong performance. We present experimental results on standard benchmarks that suggest our approach is the state-of-the-art system for pose estimation, improving past work on the challenging Parse and Buffy datasets while being orders of magnitude faster.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields (2016, background): “Human 2D pose estimation—the problem of localizing anatomical keypoints or “parts”—has largely focused on finding body parts of individuals [8, 4, 3, 21, 33, 13, 25, 31, 6, 24].”
Convolutional Pose Machines (2016, background): “Handling multiple people in a single end-to-end architecture is also a challenging problem and an interesting avenue for future work.”
Deformable part models are convolutional neural networks (2014, background): “However, the construction is general and naturally extends to a variety of models such as object detection grammars [19] and recent DPM variants, such as [44].”
Feature Boosting Network For 3D Pose Estimation (2019, methodology): “[21] described a method for articulated human detection and pose estimation in static images based on the representation of deformable part models with a tree structure.”
Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations (2014, background): “In this case, each part is associated with only one informative IDPR term, which can be merged with the appearance term of each part to define different “types” of part in [27].”
Tracking Emerges by Colorizing Videos (2018, methodology): “We use the standard PCK metric from [61] which measures the percentage of keypoints that are suﬃciently close to the ground truth.”
Flowing ConvNets for Human Pose Estimation in Videos (2015, methodology): “(2014) [24] who use a deep network similar to our CoordinateNet (with multiple input frames); and the deformable part-based model of Yang & Ramanan (2013) [39].”
Unsupervised Learning using Sequential Verification for Action Recognition (2016, methodology): “We evaluate the Probability of Correct Keypoints (PCK) measure [80] on the wrist and elbow keypoints as used in [81] (Figure 6).”
Humans in 4D: Reconstructing and Tracking Humans with Transformers (2023, neutral): “Given an input image I , the model predicts Θ = [ θ, β, π ] = f ( I ) .”