Multi-person Pose Estimation with Local Joint-to-Person Associations

By Juergen Gall. 2016

158 citations recorded by the source corpus.

Despite of the recent success of neural networks for human pose estimation, current approaches are limited to pose estimation of a single person and cannot handle humans in groups or crowds. In this work, we propose a method that estimates the poses of multiple persons in an image in which a person can be occluded by another person or might be truncated. To this end, we consider multi-person pose estimation as a joint-to-person association problem. We construct a fully connected graph from a set of detected joint candidates in an image and resolve the joint-to-person association and outlier detection using integer linear programming. Since solving joint-to-person association jointly for all persons in an image is an NP-hard problem and even approximations are expensive, we solve the problem locally for each person. On the challenging MPII Human Pose Dataset for multiple persons, our approach achieves the accuracy of a state-of-the-art method, but it is 6,000 to 19,000 times faster.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Accurate Multi-person Pose Estimation in the Wild (2017, neutral): “[26].”
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields (2016, methodology): “A common approach [28, 11, 33, 16] is to divide and conquer the problem by employing a person detector and performing single-person pose estimation for each detection.”
Associative Embedding: End-to-End Learning for Joint Detection and Grouping (2016, background): “Our novelty lies in the integration of detection and grouping into a single network; to the best of our knowledge such an integration has not been attempted for multiperson human pose estimation.”
Human Pose Regression with Residual Log-likelihood Estimation (2021, background): “In the bottom-up framework [48, 18, 20, 4, 38, 44, 8], multiple body joints are retrieved from the heatmaps and grouped into different human poses.”
HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation (2019, background): “Bottom-up methods [35, 19, 20, 3, 30] detect identity-free body joints for all the persons in an image and then group them into individuals.”
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields (2018, methodology): “Multi-Person Pose Estimation For multi-person pose estimation, most approaches [5], [6], [38], [39], [40], [41], [42], [43], [44] have used a top-down strategy that first detects people and then have estimated the pose of each person independently on each detected region.”
Single-Shot Multi-person 3D Pose Estimation from Monocular RGB (2017, background): “Multi-Person 2D Pose Estimation : A common approach for multi-person 2D pose estimation is to ﬁrst detect single persons and then 2D pose [44, 13, 55, 19, 40].”
Monocular human pose estimation: A survey of deep learning-based methods (2020, methodology): “…al., 2015; Luo et al., 2018) (5) Network compression : (Tang et al., 2018b; Debnath et al., 2018; Feng et al., 2019) 2D Multiple Top-down coarse-to-ﬁne (Iqbal and Gall, 2016; Huang et al., 2017), bounding box reﬁnement (Fang et al., 2017), multi-level feature fusion (Xiao et al., 2018; Chen et…”
PoseTrack: A Benchmark for Human Pose Estimation and Tracking (2017, background): “Human pose estimation has recently made significant progress on the tasks of single person pose estimation in individual frames [46, 45, 44, 4, 49, 15, 18, 31, 2, 36] and videos [34, 6, 21, 12] as well as multi-person pose estimation in monocular images [35, 18, 20, 3, 32].”
RMPE: Regional Multi-person Pose Estimation (2016, background): “Head Shoulder Elbow Wrist Hip Knee Ankle Total time [s/frame] Iqbal&Gall [29] 58.”
PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model (2018, unknown): “[30] formulate the problem of multi-person pose estimation as part grouping and labeling via a Linear Program.”