3D Human Pose Estimation = 2D Pose Estimation + Matching

By Ching-Hang Chen, Deva Ramanan. 2016

579 citations recorded by the source corpus.

We explore 3D human pose estimation from a single RGB image. While many approaches try to directly predict 3D pose from image measurements, we explore a simple architecture that reasons through intermediate 2D pose predictions. Our approach is based on two key observations (1) Deep neural nets have revolutionized 2D pose estimation, producing accurate 2D predictions even for poses with self-occlusions (2) Big-datasets of 3D mocap data are now readily available, making it tempting to lift predicted 2D poses to 3D through simple memorization (e.g., nearest neighbors). The resulting architecture is straightforward to implement with off-the-shelf 2D pose estimation systems and 3D mocap libraries. Importantly, we demonstratethatsuchmethodsoutperformalmostallstate-of-theart 3D pose estimation systems, most of which directly try to regress 3D pose from 2D measurements.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Feature Boosting Network For 3D Pose Estimation (2019, background): “Most of the existing works make use of highly constrained configurations [2], such as multi-view systems [3] and depth sensors [4], to infer the 3D poses.”
2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, methodology): “Chen and Ramanan [11] divided the problem of 3D pose estimation into two parts.”
Compositional Human Pose Regression (2017, methodology): “Yasin [52] Rogez [40] Chen [7] Moreno [30] Zhou [57] Baseline Ours (all)”
A model-based gait recognition method with body pose and human prior knowledge (2020, methodology): “The unique advantage of [18] is that it can estimate 3D pose from one single image, speciﬁcally from the 2D pose by human body constrains.”
Deep learning tools for the measurement of animal behavior in neuroscience (2019, background): “Yet currently, the highest accuracy is achieved by using multiple 2D views to reconstruct a 3D estimate (Figure 1b; [35,20]), but other ways of ‘lifting’ 2D into 3D are being actively explored [36,37,20].”
Integral Human Pose Regression (2017, methodology): “For evaluation, many previous works [8, 46, 32, 54, 25, 31, 37, 51, 41, 4, 53, 44, 56] use the mean per joint position error (MPJPE ).”
Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image (2019, unknown): “[3, 4, 6, 21, 30, 43, 46] are based on the two-stage approach.”
3D Human Pose Estimation in the Wild by Adversarial Learning (2018, background): “Two-stage approaches first estimate 2D poses and then lift 2D poses to 3D poses [58, 4, 2, 51, 28, 40, 25, 57, 30].”
Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision (2016, methodology): “Other approaches use the estimated 2D pose to look up a suitable 3D pose from a dictionary [12], or use the ground truth 2D pose based dictionary lookup to create 3D annotations for in-the-wild 2D pose data [48], but neither address the 2D to 3D ambiguity.”
Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation (2018, methodology): “Similar to model-based methods, learning approaches have benefited from the advent of robust 2D pose methods – by matching 2D detections to a 3D pose database [8, 66], by regressing pose from 2D joint distance matrices [35], by exploiting pose and geometric priors for lifting [69, 1, 51, 19, 32, 70,”
Learning to Estimate 3D Human Pose and Shape from a Single Color Image (2018, background): “Other approaches commit to the 2D pose estimates provided by state-of-the-art ConvNets and focus on the 3D pose reconstruction [29, 57], recover 3D pose exemplars [8], or produce multiple 3D pose candidates consistent with the 2D pose [18].”
A Simple Yet Effective Baseline for 3d Human Pose Estimation (2017, background): “[14], who incorporated temporal constraints during search, and by Chen and Ramanan [9].”
Ordinal Depth Supervision for 3D Human Pose Estimation (2018, background): “Reconstruction approaches: A long line of approaches follows the reconstruction paradigm by employing 2D pose detectors to localize 2D human joints and using these locations to estimate plausible 3D poses [10, 17].”
Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “Chen and Ramanan [118] deployed a nearest neighbor matching of the predicted 2D pose and 3D pose”
Learning to Estimate 3D Hand Pose from Single RGB Images (2017, methodology): “[6] deployed a nearest neighbor matching of a given 2D prediction using a database of 2D to 3D correspondences.”
Survey on Emotional Body Gesture Recognition (2018, methodology): “plane, some methods infer an additional depth dimension [83], [84], [85], [86].”
Semantic Graph Convolutional Networks for 3D Human Pose Regression (2019, unknown): “5 Chen & Ramanan [7] CVPR’17 89.”
Monocular Total Capture: Posing Face, Body, and Hands in the Wild (2018, methodology): “The two-stage methods take 2D keypoint estimation as input and focus on lifting 2D human poses to 3D without considering input image [9, 17, 33, 36, 39, 20].”