Reconstructing 3D Human Pose from 2D Image Landmarks

By Varun Ramakrishna, Takeo Kanade, Yaser Sheikh. 2012

338 citations recorded by the source corpus.

Reconstructing an arbitrary configuration of 3D points from their projection in an image is an ill-posed problem. When the points hold semantic meaning, such as anatomical landmarks on a body, human observers can often infer a plausible 3D configuration, drawing on extensive visual memory. We present an activity-independent method to recover the 3D configuration of a human figure from 2D locations of anatomical landmarks in a single image, leveraging a large motion capture corpus as a proxy for visual memory. Our method solves for anthropometrically regular body pose and explicitly estimates the camera via a matching pursuit algorithm operating on the image projections. Anthropometric regularity (i.e., that limbs obey known proportions) is a highly informative prior, but directly applying such constraints is intractable. Instead, we enforce a necessary condition on the sum of squared limb-lengths that can be solved for in closed form to discourage implausible configurations in 3D. We evaluate performance on a wide variety of human poses captured from different viewpoints and show generalization to novel 3D configurations and robustness to missing data.

Subfields: Computer Vision and Pattern Recognition, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

End-to-End Recovery of Human Shape and Pose (2017, methodology): “height, weight, bone ratios), and subsumes the geometric priors used by models that only predict 3D joint locations [34, 40, 51].”
Single Image 3D Interpreter Network (2016, background): “Typically, researchers recover a 3D skeleton from a single image by minimizing its projection error on the 2D image plane [26,27,28].”
A Simple, Fast and Highly-Accurate Algorithm to Recover 3D Shape from 2D Landmarks on a Single Image (2016, methodology): “Previous methods usually ﬁt a shape-space model to recover the 3D conﬁgu-ration of the object from the 2D locations of landmarks in a single image [2], [3], [13], [14].”
Exploiting Temporal Information for 3D Human Pose Estimation (2017, background): “Another common approach to estimating 3D joint locations given the 2D pose is to separate the camera pose variability from the intrinsic deformation of the human body, the latter of which is modeled by learning an over-complete dictionary of basis 3D poses from a large database of motion capture dat”
Learning to Estimate 3D Human Pose and Shape from a Single Color Image (2018, neutral): “[35]* 157.”
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose (2016, background): “Given its ill-posed nature, researchers have generally approached 3D human pose estimation in simplified settings, such as assuming background subtraction is feasible [1], relying on groundtruth 2D joint locations to estimate 3D pose [26, 43], employing additional camera views [7, 15], and capitaliz”
Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation (2018, neutral): “Mean Median Akhter & Black [1] 181.1 158.1 Ramakrishna et al. [45] 157.3 136.8 Zhou et al. [68] 106.7 90.0 SMPLify [6] 82.3 69.3 SMPLify (dense) [24] 80.7 70.0 SelfSup [62] 98.4 - Pavlakos et al. [38] 75.9 - HMR (H36M-trained)[22] 77.6 72.1 HMR [22] 56.8 -”
Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation (2016, background): “feature descriptors [58, 62, 63] and 2D joint locations [2, 3, 5, 20, 36, 51, 57, 69, 70].”
3D Human pose estimation: A review of the literature and analysis of covariates (2016, background): “[77, 78] Yes Enforce anthropometric constraints and estimate the param-”
Monocular Total Capture: Posing Face, Body, and Hands in the Wild (2018, methodology): “age) [58, 44, 4, 60, 9, 35, 33, 73, 27, 36, 69] using available 2D and 3D human pose datasets [5, 28, 1, 22, 25].”
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training (2018, background): “Before the success of deep learning, most approaches to 3D pose estimation were based on feature engineering and assumptions about skeletons and joint mobility [48, 42, 20, 18].”
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image (2016, methodology): “We compare our approach to three state-of-the-art methods [4,39,58] and also use these data for an ablation analysis.”
Robust Estimation of 3D Human Poses from a Single Image (2014, methodology): “It is similar to [12] but there are ﬁve distinctive differences.”
3D Human Pose Estimation in the Wild by Adversarial Learning (2018, neutral): “ECCV’12 [34] 137.”
Sparse Representation for 3D Shape Estimation: A Convex Relaxation Approach (2015, methodology): “The rows from top to bottom correspond to the input 2D poses, the ground-truth 3D poses (visualized in a novel view), and the reconstructions from the proposed method, the alternating minimization, and the PMP method [8], respectively.”
Using Locally Corresponding CAD Models for Dense 3D Reconstructions from a Single Image (2017, background): “With the remarkable success in Structure from Motion (SfM), which is now capable of reconstructing entire cities using large-scale photo collections [1] and real-time visual SLAM on embedded and mobile devices [14], the computer vision community is starting to explore the possibility of constructing”
MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior (2017, neutral): “, [5], [58]), the proposed approach does not rely on a hard decision of 2D correspondences before reconstruction and”
Sim2real transfer learning for 3D human pose estimation: motion to the rescue (2019, methodology): “One straightforward approach is to ‘lift’ 2D poses into 3D, using either dictionaries or direct regression [3, 44, 47, 64, 79, 85, 88, 93].”
Towards 3D Human Pose Estimation in the Wild: A Weakly-Supervised Approach (2017, methodology): “The second step recovers a 3D pose from these 2D joints [21, 32, 1].”