Fusing 2D Uncertainty and 3D Cues for Monocular Body Pose Estimation

By Bugra Tekin, Pablo Márquez-Neila, Mathieu Salzmann, Pascal Fua. 2016

36 citations recorded by the source corpus.

Most recent approaches to monocular 3D human pose estimation rely on Deep Learning. They typically involve training a network to regress from an image to either 3D joint coordinates directly, or 2D joint locations from which the 3D coordinates are inferred by a model-fitting procedure. The former takes advantage of 3D cues present in the images but rarely models uncertainty. By contrast, the latter often models 2D uncertainty, for example in the form of joint location heatmaps, but discards all the image information, such as texture, shading and depth cues, in the fitting step. In this paper, we therefore propose to jointly model 2D uncertainty and leverage 3D image cues in a regression framework for monocular 3D human pose estimation. To this end, we introduce a novel two-stream deep architecture. One stream focuses on modeling uncertainty via probability maps of 2D joint locations and the other exploits 3D cues by directly acting on the image. We then study different approaches to fusing their outputs to obtain the final 3D prediction. Our experiments evidence in particular that our late-fusion mechanism improves upon the state-of-the-art by a large margin on standard 3D human pose estimation benchmarks.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, background): “Recently, deep architectures have been used to learn precise 3D representations from RGB images [60, 50, 30, 49, 31, 39], thanks to the availability of high quality data [21], and are now able to surpass depthsensors [32].”
Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image (2017, unknown): “[36] 85.”
VNect (2017, methodology): “[2016c], where batches of frames are processed offline after motion compensation, and is related to the recently proposed per-frame combination of 2D and 3D pose [Tekin et al. 2016b].”
Total Capture: 3D Human Pose Estimation Fusing Video and Inertial Sensors (2017, methodology): “We use the standard evaluation protocol as followed by [16, 19, 28, 29, 30] where subjects S1, S5, S6, S7, S8 are used for training and Subjects S9, S11 provide the test sequences.”
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision (2016, methodology): “Pretraining on ImageNet [49] is commonly used for weight initialization [22, 61] in CNNs.”