Robust Estimation of 3D Human Poses from a Single Image

By Chunyu Wang, Yizhou Wang, Zhouchen Lin, Alan L. Yuille, Wen Gao. 2014

219 citations recorded by the source corpus.

Human pose estimation is a key step to action recognition. We propose a method of estimating 3D human poses from a single image, which works in conjunction with an existing 2D pose/joint detector. 3D pose estimation is challenging because multiple 3D poses may correspond to the same 2D pose after projection due to the lack of depth information. Moreover, current 2D pose estimators are usually inaccurate which may cause errors in the 3D estimation. We address the challenges in three ways: (i) We represent a 3D pose as a linear combination of a sparse set of bases learned from 3D human skeletons. (ii) We enforce limb length constraints to eliminate anthropomorphically implausible skeletons. (iii) We estimate a 3D pose by minimizing the 1-norm error between the projection of the 3D pose and the corresponding 2D detection. The 1-norm loss term is robust to inaccurate 2D joint estimations. We use the alternating direction method (ADM) to solve the optimization problem efficiently. Our approach outperforms the state-of-the-arts on three benchmark datasets.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Space-Time Representation of People Based on 3D Skeletal Data: A Review (2016, methodology): “[65] constructs a 3D human skeleton from a single image using a linear combination of known skeletons with physical constraints on limb lengths.”
3D Human Pose Estimation = 2D Pose Estimation + Matching (2016, background): “This is classically treated as a constrained optimization problem who’s objective minimizes the 2D reprojection error of an unknown 3D pose and unknown camera [37, 32, 24, 2].”
Using Locally Corresponding CAD Models for Dense 3D Reconstructions from a Single Image (2017, background): “With the remarkable success in Structure from Motion (SfM), which is now capable of reconstructing entire cities using large-scale photo collections [1] and real-time visual SLAM on embedded and mobile devices [14], the computer vision community is starting to explore the possibility of constructing”
Bayesian Image Based 3D Pose Estimation (2016, methodology): “In our approach, instead, as in [11,12], body parts in 2D are detected by using the algorithm introduced in [2].”
Synthesizing Training Images for Boosting Human 3D Pose Estimation (2016, methodology): “Since all of the above methods estimate 2D poses, several methods have been proposed for recovering 3D joints from their 2D locations [45, 40, 12, 4, 57, 55, 52].”
LCR-Net++: Multi-Person 2D and 3D Pose Detection in Natural Images (2018, background): “large body of work tackles 3D pose estimation from 2D poses assuming that the 2D joints are available [24], [25], provided by an off-the-shelf 2D pose detector [26], [27], [28], [29], [30], [31], [32], [33] or obtained through a 2D pose estimationmodule within the proposed architecture [34], [35].”
3D Human pose estimation: A review of the literature and analysis of covariates (2016, background): “This behavior can be attributed to the fact that regression-based techniques like the Weighted KNN or the Twin Gaussian Processes KNN rely on image-based features (i.e., HoGs), whereas the methods of Wang et al. [70] and Zhou et al. [72] require an initial model and a skeleton structure that tend to”
Semantic Graph Convolutional Networks for 3D Human Pose Regression (2019, methodology): “Other methods created over-complete bases which are suitable for representing human poses as sparse combinations [2, 4, 44, 62, 77].”
Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera (2018, background): “Several works lift 2D detections to 3D using learning or geometric reasoning [18, 29, 35, 9, 26, 49, 33, 48, 44, 34, 13, 43, 45].”
Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection (2019, unknown): “[44] investigate local methods for shape alignment.”
Deep Multitask Architecture for Integrated 2D and 3D Human Sensing (2017, methodology): “There is also interest in combining 2d and 3d estimation methods in order to obtain models capable of multiple task, and able to operate in realistic imaging conditions [41, 32, 47, 2].”
Sim2real transfer learning for 3D human pose estimation: motion to the rescue (2019, methodology): “One straightforward approach is to ‘lift’ 2D poses into 3D, using either dictionaries or direct regression [3, 44, 47, 64, 79, 85, 88, 93].”
MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild (2016, methodology): “Some approaches use as input the 2D joints automatically provided by a 2D pose detector [32, 38], while others jointly solve the 2D and 3D pose estimation [31, 43].”
Monocular 3D Human Pose Estimation Using Transfer Learning and Improved CNN Supervision (2016, background): “A common approach is to lift 2D keypoints to 3D [84, 11, 81, 41, 88, 91, 87, 69, 68], but this requires computationally expensive iterative pose optimization which may be unstable under depth ambiguities.”
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image (2017, methodology): “Representing human 3D pose as a linear combination of a sparse set of 3D bases, pretrained using 3D mocap data, has also proved a popular approach for articulated human motion [28, 43, 49], while [49] propose a convex relaxation to jointly estimate the coefficients of the sparse representation and t”
Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video (2015, methodology): “Much of this work has been realized by assuming manually labeled 2D joint locations; however, there is some recent work that has used a 2D pose detector to automatically provide the input joints [40, 48] or solved 2D and 3D pose estimation jointly [39, 54].”