A Simple Yet Effective Baseline for 3d Human Pose Estimation

By Julieta Martinez, Rayat Hossain, Javier Romero, James J. Little. 2017

1,518 citations recorded by the source corpus.

Following the success of deep convolutional networks, state-of-the-art methods for 3d human pose estimation have focused on deep end-to-end systems that predict 3d joint locations given raw image pixels. Despite their excellent performance, it is often not easy to understand whether their remaining error stems from a limited 2d pose (visual) understanding, or from a failure to map 2d poses into 3- dimensional positions.,,With the goal of understanding these sources of error, we set out to build a system that given 2d joint locations predicts 3d positions. Much to our surprise, we have found that, with current technology, "lifting" ground truth 2d joint locations to 3d space is a task that can be solved with a remarkably low error rate: a relatively simple deep feedforward network outperforms the best reported result by about 30% on Human3.6M, the largest publicly available 3d pose estimation benchmark. Furthermore, training our system on the output of an off-the-shelf state-of-the-art 2d detector (i.e., using images as input) yields state of the art results – this includes an array of systems that have been trained end-to-end specifically for this task. Our results indicate that a large portion of the error of modern deep 3d pose estimation systems stems from their visual analysis, and suggests directions to further advance the state of the art in 3d human pose estimation.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, background): “Recently, deep architectures have been used to learn precise 3D representations from RGB images [60, 50, 30, 49, 31, 39], thanks to the availability of high quality data [21], and are now able to surpass depthsensors [32].”
End-to-End Recovery of Human Shape and Pose (2017, methodology): “Two stage methods first predict 2D joint locations using 2D pose detectors [30, 49, 54] or ground truth 2D pose and then predict 3D joint locations from the 2D joints either by regression [26, 29] or model fitting, where a common approach exploits a learned dictionary of 3D skeletons [2, 34, 47, 37,”
Compositional Human Pose Regression (2017, methodology): “2D joints are first estimated and then used to estimate the 3D pose via learning a 2D to 3D mapping (Moreno-Noguer, 2016; Martinez et al., 2017; Fang et al., 2018), fitting a 3D model via optimization (Zhou et al.”
Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks (2019, methodology): “[28] directly regressed 3D keypoints from extracted 2D poses via a simple network composed of several fully-connected layers.”
Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes: The Importance of Multiple Scene Constraints (2018, methodology): “Our work relates to recently developed deep architectures for 2d human pose estimation [4, 9, 21, 35, 36], 3d human pose estimation based on fitting volumetric models [2, 15], feedforward deep models for 3d prediction [18, 22, 40], as well as integrated deep models for 2d and 3d reasoning [23, 27, 3”
Human Pose Regression with Residual Log-likelihood Estimation (2021, methodology): “Two-stage methods [32, 13, 47, 68, 60, 9, 30, 67] lift the 2D poses to 3D space by regression.”
A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose (2021, background): “While feed-forward estimation of the 3D joint positions [29, 35, 37, 38, 46, 47, 55, 62, 67, 73, 76] or joint angles and bone lengths [56, 77] of the skeleton is highly accurate, such discriminative estimates are prone to misalignment when overlayed onto the input image due to the generalization gap”
GHUM & GHUML: Generative 3D Human Shape and Articulated Pose Models (2020, background): “Many image and video-based pose and shape estimation methods have also been proposed[33, 27, 25, 40, 22, 1, 12, 26, 34, 23].”
Learning 3D Human Dynamics From Video (2018, background): “It is encouraging to see that not just 2D but also 3D pose improves from pseudo-ground truth 2D pose annotations.”
Self-Supervised Learning of 3D Human Pose Using Multi-View Geometry (2019, methodology): “In the literature there are several techniques [22, 11, 39] to lift detected 2D keypoints into 3D joints.”
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose (2020, methodology): “The architecture of the PoseNet is based on that of [10, 39].”
You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions (2019, background): “This is a well-known issue in motion forecasting: doing “nothing” can be better than doing something, since the average standing pose is in-between many test poses, representing a “safe” estimate [38].”
Semantic Graph Convolutional Networks for 3D Human Pose Regression (2019, methodology): “In order to be in line with the setting of previous works [13, 34], we employ HourGlass [38] (HG) as the 2D detector.”
Accurate 3D Human Pose in The Wild Using IMUs and a Moving Camera (2018, neutral): “For every example we show from left to right: a cropped image of the person to be estimated, our results, method [3], method [4] and method [2].”
DLow: Diversifying Latent Flows for Diverse Human Motion Prediction (2020, methodology): “Following previous work [53, 49, 73, 56], we adopt a 17-joint skeleton and train on five subjects (S1, S5, S6, S7, S8) and test on two subjects (S9 and S11).”
MotionBERT: A Unified Perspective on Learning Human Motion Representations (2022, methodology): “Following previous works [73, 83], we use subjects 1, 5, 6, 7, 8 for training, and subjects 9, 11 for testing.”
Recovering Accurate 3D Human Pose in the Wild Using IMUs and a Moving Camera (2018, background): “Several works lift 2D detections to 3D using learning or geometric reasoning [18, 29, 35, 9, 26, 49, 33, 48, 44, 34, 13, 43, 45].”
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion (2019, methodology): “Besides the fully supervised methods [25, 26], several works have explored multi-view supervision [20, 29, 31], ordinal depth supervision [28], unpaired 2D-3D data [30, 36, 41, 15] or videos [17] to alleviate the need for full 2D-3D annotations.”
Deep 3D human pose estimation: A review (2021, positive): “For example, Martinez et al. (2017) propose a simple baseline focusing on lifting 2D poses to 3D with a simple yet very effective neural network, which popularizes the research on lifting 2D pose to 3D space.”
Single-Shot Multi-person 3D Pose Estimation from Monocular RGB (2017, background): “[29] (62.”