3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information

By Sungheon Park, Jihye Hwang, Nojun Kwak. 2016

172 citations recorded by the source corpus.

While there has been a success in 2D human pose estimation with convolutional neural networks (CNNs), 3D human pose estimation has not been thoroughly studied. In this paper, we tackle the 3D human pose estimation task with end-to-end learning using CNNs. Relative 3D positions between one joint and the other joints are learned via CNNs. The proposed method improves the performance of CNN with two novel ideas. First, we added 2D pose information to estimate a 3D pose from an image by concatenating 2D pose estimation result with the features from an image. Second, we have found that more accurate 3D poses are obtained by combining information on relative positions with respect to multiple joints, instead of just one root joint. Experimental results show that the proposed method achieves comparable performance to the state-of-the-art methods on Human 3.6m dataset.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Feature Boosting Network For 3D Pose Estimation (2019, methodology): “[18] introduced a CNN framework by simultaneously training for both 2D joint classification and 3D joint regression.”
Compositional Human Pose Regression (2017, methodology): “Most previous regression based methods [6, 54, 33, 42, 43] directly minimize the squared difference of the predicted and ground truth joints.”
A Simple Yet Effective Baseline for 3d Human Pose Estimation (2017, methodology): “2d/3d positions Our first design choice is to use 2d and 3d points as inputs and outputs, in contrast to recent work that has used raw images [11,13,24,32,33,45,46,54,56] or 2d probability distributions [33, 56] as inputs, and 3d probabilities [33], 3d motion parameters [54] or basis pose coefficien”
Deep 3D human pose estimation: A review (2021, unknown): “In order to help the model to discard unnatural human 3D joint positions, Park et al. (2016) come up with an new solution which is concatenating 2D pose estimation results and image features.”
LCR-Net++: Multi-Person 2D and 3D Pose Detection in Natural Images (2018, methodology): “CNNs have been used for full-body pose estimation both in regression [7], [10], [11], [12], [13] and classification [8] approaches.”
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training (2018, background): “Some approaches leverage both image features and 2D ground-truth poses [39, 41, 52, 54].”
Exploiting Temporal Information for 3D Human Pose Estimation (2017, methodology): “[14] designed CNNs to jointly predict 2D and 3D poses.”
A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose (2021, background): “While feed-forward estimation of the 3D joint positions [29, 35, 37, 38, 46, 47, 55, 62, 67, 73, 76] or joint angles and bone lengths [56, 77] of the skeleton is highly accurate, such discriminative estimates are prone to misalignment when overlayed onto the input image due to the generalization gap”
Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation (2016, methodology): “In Table 1, we compare the results of our trainable fusion approach with those of the following stateof-the-art single image-based methods: KDE regression from HOG features to 3D poses [30], jointly training a 2D body part detector and a 3D pose regressor [38, 45], the maximum-margin structured lear”
MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior (2017, unknown): “F”
Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks (2019, methodology): “[36] introduced an end-to-end framework with simultaneous training of both 2D joint classification and 3D joint regression.”
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose (2016, background): “41mm for coordinate regression is comparable to recent reported results with a coordinate regression target output [19, 35, 22, 44].”
Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image (2019, methodology): “[3, 4, 6, 21, 30, 43, 46] are based on the two-stage approach.”
HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation (2020, methodology): “Two-stage approaches ﬁrst estimate 2D pose and then lift them to 3D joint locations by a learned dictionary of 3D skeleton [1, 55, 66, 58, 78, 79] or regression [48, 73, 42, 11, 34, 61].”