3D Pictorial Structures for Multiple Human Pose Estimation

By Vasileios Belagiannis, Sikandar Amin, Mykhaylo Andriluka, Bernt Schiele, Nassir Navab, Slobodan Ilic. 2014

280 citations recorded by the source corpus.

In this work, we address the problem of 3D pose estimation of multiple humans from multiple views. This is a more challenging problem than single human 3D pose estimation due to the much larger state space, partial occlusions as well as across view ambiguities when not knowing the identity of the humans in advance. To address these problems, we first create a reduced state space by triangulation of corresponding body joints obtained from part detectors in pairs of camera views. In order to resolve the ambiguities of wrong and mixed body parts of multiple humans after triangulation and also those coming from false positive body part detections, we introduce a novel 3D pictorial structures (3DPS) model. Our model infers 3D human body configurations from our reduced state space. The 3DPS model is generic and applicable to both single and multiple human pose estimation. In order to compare to the state-of-the art, we first evaluate our method on single human 3D pose estimation on HumanEva-I [22] and KTH Multiview Football Dataset II [8] datasets. Then, we introduce and evaluate our method on two datasets for multiple human 3D pose estimation. In order to compare to the state-of-the art, we first evaluate our method on single human 3D pose estimation on HumanEva-I [22] and KTH Multiview Football Dataset II [8] datasets. Then, we introduce and evaluate our method on two datasets for multiple human 3D pose estimation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Towards Accurate Multi-person Pose Estimation in the Wild (2017, methodology): “D person detection is used as a ﬁrst step in several 3D pose estimation works [39, 4, 5].”
Space-Time Representation of People Based on 3D Skeletal Data: A Review (2016, background): “Among a large number of human representation approaches [24], [25], [26], [27], [28], [29], most of the existing 3D based methods can be broadly grouped into two categories: representations based on local features [30], [31] and skeletonbased representations [32] [33] [34].”
DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation (2015, background): “[13] proposes a multi-person PS model that explicitly models depth ordering and person-person occlusions.”
Recurrent Human Pose Estimation (2016, methodology): “Inspired by the idea of part-based models [2], [3], we additionally propose body part heatmaps which are constructed by pairs of keypoints.”
VNect (2017, background): “2009; Johnson and Everingham 2010] and have been extended to 3D pose estimation [Amin et al. 2013; Balan et al. 2007; Belagiannis et al. 2014; Sigal et al. 2012].”
Self-Supervised Learning of 3D Human Pose Using Multi-View Geometry (2019, methodology): “Early work [1, 5, 6, 3, 4] used 2D pose estimations obtained from calibrated cameras to produce 3D pose by triangulation or pictorial structures model.”
Monocular 3D Human Pose Estimation Using Transfer Learning and Improved CNN Supervision (2016, background): “Combinig discriminative and generative methods yields increased robustness outdoors [70, 67, 3, 8, 21, 55, 56, 11].”
Efficient ConvNet-based marker-less motion capture in general scenes with a low number of cameras (2015, background): “Motion capture has many applications, for instance in sports, biomedical research, or computer animation.”
A Review of Human Activity Recognition Methods (2015, background): “Belagiannis et al. (2014) have also addressed the problem of multiview pose estimation.”
Panoptic Studio: A Massively Multiview System for Social Motion Capture (2015, methodology): “To be scalable to a large number of participants, our method avoids subject-speciﬁc templates such as body shape, color, and bone length, while providing high accuracy without jitter.”
3D Human Pose Estimation from a Single Image via Distance Matrix Regression (2016, methodology): “This is the same evaluation protocol used by the baselines we compare against [3, 6, 8, 10, 12, 21, 30, 31, 38, 39, 40, 45, 47, 50, 52, 55, 56].”
Deep 3D human pose estimation: A review (2021, unknown): “The Campus dataset has annotated the body joints of the main three actors performing different actions for the frames that are observed from the first two cameras.”
Panoptic Studio: A Massively Multiview System for Social Interaction Capture (2016, methodology): “A few methods facilitate body pose detectors in multiple views to reconstruct 3D body poses [46], [47], [48], [49], [50].”
3D Human pose estimation: A review of the literature and analysis of covariates (2016, methodology): “A few methods [46, 149] have tried to address this limitation, by providing datasets (along with 3D annotation of the joints) in outdoor environments but they are far from simulating effectively reallife conditions.”
Robust Optimization for Deep Regression (2015, background): “It has been extensively studied from the single-and multi-view perspective, where the standard ways to tackle the problem are based on part-based models [2, 4, 11, 32, 37, 46] and holistic approaches [8, 12, 28, 33].”
Fusing 2D Uncertainty and 3D Cues for Monocular Body Pose Estimation (2016, methodology): “On the KTH Multiview Football II dataset, we evaluate our method on the sequence containing Player 2, as in [6, 9, 59].”
Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes: The Importance of Multiple Scene Constraints (2018, background): “Systems designed for the 3d reconstruction of multiple people are relatively rare and existing ones are based on multiple cameras [1, 6, 5, 12, 14].”
Deep Kinematic Pose Regression (2016, background): “Its extensions [15,36,24] achieve promising results in 2D human estimation, and has been extended to 3D human pose [2].”