Structured Prediction of 3D Human Pose with Deep Neural Networks

By Bugra Tekin, Isinsu Katircioglu, Mathieu Salzmann, Vincent Lepetit, Pascal Fua. 2016

280 citations recorded by the source corpus.

Most recent approaches to monocular 3D pose estimation rely on Deep Learning. They either train a Convolutional Neural Network to directly regress from image to 3D pose, which ignores the dependencies between human joints, or model these dependencies via a max-margin structured learning framework, which involves a high computational cost at inference time. In this paper, we introduce a Deep Learning regression architecture for structured prediction of 3D human pose from monocular images that relies on an overcomplete autoencoder to learn a high-dimensional latent pose representation and account for joint dependencies. We demonstrate that our approach outperforms state-of-the-art ones both in terms of structure preservation and prediction accuracy.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Feature Boosting Network For 3D Pose Estimation (2019, methodology): “[17] used auto-encoders to learn structured latent representations for 3D pose regression”
3D Human Pose Estimation from a Single Image via Distance Matrix Regression (2016, methodology): “Early discriminative approaches [1, 9, 32, 36, 44] focused on directly predicting 3D pose from image descriptors such as SIFT of HOG filters, and more recently, from rich features encoding body part information [16] and from the entire image in Deep architectures [23, 46].”
LCR-Net++: Multi-Person 2D and 3D Pose Detection in Natural Images (2018, background): “Recently, this has been naturally extended to end-to-end mappings using CNN architectures, either in monocular images [7], [8], [12], [41], [42], [43] or in videos [11], [44].”
3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information (2016, methodology): “Moreover, from Table 1, it is justiﬁed that our method outperforms the CNN based methods that predict 3D pose from a single image [5,6,7].”
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction (2019, background): “There is rich recent literature on 3D pose estimation in the form of a simplistic body skeleton, e.g., [3, 19, 22, 24, 25, 29, 30, 34, 35, 38, 40, 41, 42, 50, 51].”
Single-Shot Multi-person 3D Pose Estimation from Monocular RGB (2017, background): “Single-Person 3D Pose Estimation : Existing monocular single-person 3D pose methods show good performance on standard datasets [18, 51, 59, 41, 28, 27].”
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose (2016, methodology): “41mm for coordinate regression is comparable to recent reported results with a coordinate regression target output [19, 35, 22, 44].”
Weakly-supervised Transfer for 3D Human Pose Estimation in the Wild (2017, background): “[25] embed an pre-trained auto-encoder at the top of the network.”
Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation (2016, methodology): “Unsurprisingly , the more recent methods tend to rely on Deep Networks [38] [64] [65] [75].”
3D Human pose estimation: A review of the literature and analysis of covariates (2016, methodology): “Finally, the task of estimating the 3D human pose from image sequences has also been explored using deep learning, [98, 99, 100, 101, 102, 103] and the respective methods are going to be discussed individually in Sections 4.”
Learning to Estimate 3D Human Pose and Shape from a Single Color Image (2018, background): “Many recent works follow the end-to-end paradigm [48, 40, 42, 46, 55], using images as input to predict 3D joint locations [23, 45, 34, 28], regress 3D heatmaps [31], or classify the image in a particular pose class [39, 40].”
Monocular 3D Human Pose Estimation Using Transfer Learning and Improved CNN Supervision (2016, methodology): “It has a notably higher accuracy than known methods from the literature [11, 75, 40].”
Self-Supervised Learning of 3D Human Pose Using Multi-View Geometry (2019, methodology): “Single-view methods In many recent works, convolutional neural networks (CNN) are used to estimate the co-ordinates of the 3D joints directly from images [38, 39, 40, 35, 23].”
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training (2018, methodology): “The first neural methods with convolutional neural networks (CNN) focused on end-to-end reconstruction [28, 53, 51, 41] by directly estimating 3D poses from RGB images without intermediate supervision.”
Indirect deep structured learning for 3D human body shape and pose prediction (2017, background): “(variational [7] or not) for face rendering [8], human pose prediction [18] and general multi-class segmentation [4].”
Towards 3D Human Pose Estimation in the Wild: A Weakly-Supervised Approach (2017, background): “[25] embed a pretrained auto-encoder at the top of the network.”
Deep Learning-based Human Pose Estimation: A Survey (2020, background): “, [112] [113] [114] [43] [115] [116] [117].”
LCR-Net: Localization-Classification-Regression for Human Pose (2017, background): “Some of the competing methods on P2 only evaluate on 6 actions [14, 15, 16, 28], other leverage temporal information [7, 29, 37].”
3D Human Pose Estimation in the Wild by Adversarial Learning (2018, background): “[37] further adopted an autoencoder at the end of the network.”