3D Human Pose Estimation in the Wild by Adversarial Learning

By Wei Yang, Wanli Ouyang, X. Wang, Jimmy S. J. Ren, Hongsheng Li, Xiaogang Wang. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

395 citations recorded by the source corpus.

Recently, remarkable advances have been achieved in 3D human pose estimation from monocular images because of the powerful Deep Convolutional Neural Networks (DCNNs). Despite their success on large-scale datasets collected in the constrained lab environment, it is difficult to obtain the 3D pose annotations for in-the-wild images. Therefore, 3D human pose estimation in the wild is still a challenge. In this paper, we propose an adversarial learning framework, which distills the 3D human pose structures learned from the fully annotated dataset to in-the-wild images with only 2D pose annotations. Instead of defining hard-coded rules to constrain the pose estimation results, we design a novel multi-source discriminator to distinguish the predicted 3D poses from the ground-truth, which helps to enforce the pose estimator to generate anthropometrically valid poses even with images in the wild. We also observe that a carefully designed information source for the discriminator is essential to boost the performance. Thus, we design a geometric descriptor, which computes the pairwise relative locations and distances between body joints, as a new information source for the discriminator. The efficacy of our adversarial learning framework with the new geometric descriptor has been demonstrated through extensive experiments on widely used public benchmarks. Our approach significantly improves the performance compared with previous state-of-the-art approaches.

Subfields: Computer Vision and Pattern Recognition, Endocrinology, Diabetes and Metabolism

View Paper: https://www.semanticscholar.org/paper/02cb44528213f4d5bd38fb2dcb59002680301e8f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Endocrinology, Diabetes and Metabolism

Citations in the corpus

Semantic Graph Convolutional Networks for 3D Human Pose Regression (2019, methodology): “Other methods use synthetic datasets which are generated from deforming a human template model with the ground truth [8, 42, 48] or introduce loss functions involving high-level knowledge [40, 53, 69] in addition to joints.”
Survey on Deep Neural Networks in Speech and Vision Systems (2019, positive): “Adversarial learning is applied to 2D images in [111] to extend the output pose prediction into 3D space.”
Deep 3D human pose estimation: A review (2021, neutral): “Subsequently, a multi-source discriminator is trained in Yang et al. (2018), which takes the original image, the geometric descriptor on image-pose correspondence, the 2D pose, and depth heatmaps as input.”
Monocular human pose estimation: A survey of deep learning-based methods (2020, methodology): “…Zhou et al., 2017; Tekin et al., 2017; Li and Lee, 2019; Qammaz and Argyros, 2019; Chen and Ramanan, 2017; Moreno-Noguer, 2017; Wang et al., 2018a; Yang et al., 2018) Model-based (1) SMPL-based : (Bogo et al., 2016; Tan et al., 2017; Pavlakos et al., 2018b; Omran et al., 2018; Varol et al., 2018;…”
DeepHuman: 3D Human Reconstruction From a Single Image (2019, background): “The research can be categorized into sparse 2D parsing (2D skeleton estimation) [6, 39], sparse 3D parsing (3D skeleton estimation) [33, 44, 64, 48, 50, 35, 61], dense 2D parsing [17] and dense 3D parsing (shape and pose estimation).”
Monocular Total Capture: Posing Face, Body, and Hands in the Wild (2018, methodology): “Some work [27, 70] uses Generative Adversarial Networks (GAN) to exploit human pose prior in a data-driven manner.”
DLow: Diversifying Latent Flows for Diverse Human Motion Prediction (2020, methodology): “Following previous work [53, 49, 73, 56], we adopt a 17-joint skeleton and train on five subjects (S1, S5, S6, S7, S8) and test on two subjects (S9 and S11).”
Single-Shot Multi-person 3D Pose Estimation from Monocular RGB (2017, background): “Another approach is to train a network with separate 2D and 3D losses for the diﬀer-ent data sources [46, 68, 56, 65, 31].”
From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation (2021, background): “In addition to feature level alignment, other works also try to reduce the domain shift in the input [16] or output level [32, 40].”
GHUM & GHUML: Generative 3D Human Shape and Articulated Pose Models (2020, background): “Many image and video-based pose and shape estimation methods have also been proposed[33, 27, 25, 40, 22, 1, 12, 26, 34, 23].”
Deep Learning-based Human Pose Estimation: A Survey (2020, background): “Recent works [243, 272, 309] revealed the poor generalization of models trained with biased datasets by cross-dataset inference.”
Through-Wall Human Pose Estimation Using Radio Signals (2018, background): “It has recently witnessed major advances and significant performance improvements [30, 27, 28, 46, 31, 20, 10, 16, 33, 12, 47, 37, 45, 13].”
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training (2018, methodology): “(+) extra data – [50, 40, 58, 33] use 2D annotations from the MPII dataset, [40] uses additional data from the Leeds Sports Pose (LSP) dataset as well as ordinal annotations.”
Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19 (2020, background): “Much recent work [28]–[30] has focused on estimating the 2D [31]–[36] or 3D keypoint locations [29], [37]–[40] on the patient body.”
Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image (2019, methodology): “Most of the previous 3D human pose estimation methods [21, 32, 37, 38, 43, 46] are designed for single-person case.”