Learning Camera Viewpoint Using CNN to Improve 3D Body Pose Estimation

By Mona Fathollahi Ghezelghieh, Rangachar Kasturi, Sudeep Sarkar. 2016

57 citations recorded by the source corpus.

The objective of this work is to estimate 3D human pose from a single RGB image. Extracting image representations which incorporate both spatial relation of body parts and their relative depth plays an essential role in accurate3D pose reconstruction. In this paper, for the first time, we show that camera viewpoint in combination to 2D joint locations significantly improves 3D pose accuracy without the explicit use of perspective geometry mathematical models. To this end, we train a deep Convolutional Neural Net-work (CNN) to learn categorical camera viewpoint. To make the network robust against clothing and body shape of the subject in the image, we utilized 3D computer rendering to synthesize additional training images. We test our framework on the largest 3D pose estimation bench-mark, Human3.6m, and achieve up to 20% error reduction on standing-pose activities compared to the state-of-the-art approaches that do not use body part segmentation.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

DensePose: Dense Human Pose Estimation in the Wild (2018, background): “Synthetic ground-truth can be established by rendering images using surface-based models [33, 32, 37, 11, 5, 30].”
Feature Boosting Network For 3D Pose Estimation (2019, background): “[19] proposed to learn the camera viewpoint based on CNNs to improve the performance of 3D body pose estimation.”
Sim2real transfer learning for 3D human pose estimation: motion to the rescue (2019, background): “Principal among these is work on using datasets of synthetic humans for human pose estimation [11, 16, 20, 51, 67, 75, 84, 86, 97].”
Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose (2016, methodology): “[13] employ viewpoint prediction as a side task to provide the network with global joint configuration information.”
Configurable 3D Scene Synthesis and 2D Image Rendering with Per-pixel Ground Truth Using Stochastic Grammars (2017, background): “…2012; Romero et al. 2015; Qiu 2016), 3D human pose estimation (Shotton et al. 2013; Shakhnarovich et al. 2003; Yasin et al. 2016; Du et al. 2016; Ghezelghieh et al. 2016; Rogez and Schmid 2016; Zhou et al. 2016; Chen et al. 2016; Varol et al. 2017), depth prediction (Su et al. 2014), pedestrian…”
A Simple Yet Effective Baseline for 3d Human Pose Estimation (2017, methodology): “2d/3d positions Our first design choice is to use 2d and 3d points as inputs and outputs, in contrast to recent work that has used raw images [11,13,24,32,33,45,46,54,56] or 2d probability distributions [33, 56] as inputs, and 3d probabilities [33], 3d motion parameters [54] or basis pose coefficien”
Learning from Synthetic Humans (2017, background): “[14] render synthetic images with 10 simple body models with an emphasis on upright people; however, the main challenge using existing MoCap data for training is to generalize to poses that are not upright.”