Ordinal Depth Supervision for 3D Human Pose Estimation

By Georgios Pavlakos, Xiaowei Zhou, Kostas Daniilidis. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

408 citations recorded by the source corpus.

Our ability to train end-to-end systems for 3D human pose estimation from single images is currently constrained by the limited availability of 3D annotations for natural images. Most datasets are captured using Motion Capture (MoCap) systems in a studio setting and it is difficult to reach the variability of 2D human pose datasets, like MPII or LSP. To alleviate the need for accurate 3D ground truth, we propose to use a weaker supervision signal provided by the ordinal depths of human joints. This information can be acquired by human annotators for a wide range of images and poses. We showcase the effectiveness and flexibility of training Convolutional Networks (ConvNets) with these ordinal relations in different settings, always achieving competitive performance with ConvNets trained with accurate 3D joint coordinates. Additionally, to demonstrate the potential of the approach, we augment the popular LSP and MPII datasets with ordinal depth annotations. This extension allows us to present quantitative and qualitative evaluation in non-studio conditions. Simultaneously, these ordinal annotations can be easily incorporated in the training procedure of typical ConvNets for 3D human pose. Through this inclusion we achieve new state-of-the-art performance for the relevant benchmarks and validate the effectiveness of ordinal depth supervision for 3D human pose.

Subfields: Computer Vision and Pattern Recognition, Endocrinology, Diabetes and Metabolism, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/75512a0c23866cc2f0bcf349b770f0fb2ce17e57

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Endocrinology, Diabetes and Metabolism

In Human-Computer Interaction

Citations in the corpus

Monocular Total Capture: Posing Face, Body, and Hands in the Wild (2018, background): “Some work [41, 48, 67] proposes to use ordinal depth as additional supervision for CNN training.”
Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation (2018, result): “[36] take another approach by relying on weak 3D supervision in form of a relative 3D ordering of joints, similar to the previously proposed PoseBits [40].”
Sim2real transfer learning for 3D human pose estimation: motion to the rescue (2019, background): “Other works regress poses directly from pixels [46, 55, 56, 68, 71, 77, 78, 96], which generally relies on having a good match between training and testing.”
C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion (2019, methodology): “Besides the fully supervised methods [25, 26], several works have explored multi-view supervision [20, 29, 31], ordinal depth supervision [28], unpaired 2D-3D data [30, 36, 41, 15] or videos [17] to alleviate the need for full 2D-3D annotations.”
Monocular human pose estimation: A survey of deep learning-based methods (2020, methodology): “6M is calculated after the alignment of predictions and ground truth with a rigid transformation using Procrustes Analysis (Gower, 1975), which is also called reconstruction error (Kanazawa et al., 2018; Pavlakos et al., 2018b), P-MPJPE (Rhodin et al., 2018a) or PA-MPJPE (Sun et al., 2018).”
Deep Learning-based Human Pose Estimation: A Survey (2020, background): “, [112] [113] [114] [43] [115] [116] [117].”
Deep 3D human pose estimation: A review (2021, positive): “Pavlakos et al. (2018b) train a network called PosePrior that takes heatmaps as input and outputs pose parameters of the SMPL model and another network to estimate the shape parameters from the silhouette.”
Self-Supervised Learning of 3D Human Pose Using Multi-View Geometry (2019, background): “[28] proposed the idea of using ordinal depth relations among joints to bypass the need for full 3D supervision.”
On Self-Contact and Human Pose (2021, background): “textual and ordinal descriptions of body pose [35, 36].”
Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild (2020, methodology): “[30] employ an adversarial prior on pose, [52] explore using ordinal supervision, and [51] use texture consistency.”
3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training (2018, methodology): “Compared to [41, 40], we do not use heatmaps and instead describe poses with detected keypoint coordinates.”
Convolutional Mesh Regression for Single-Image Human Shape Reconstruction (2019, methodology): “And in fact, all the above model-based approaches underperfom in pose estimation metrics compared to approaches regressing a less informative, yet more accurate, 3D skeleton through 3D joint regression [3, 24, 29, 38].”
3D Human Pose Estimation with Spatial and Temporal Transformers (2021, methodology): “Direct estimation methods [31, 29] infer a 3D human pose from 2D images or video frames without intermediately estimating the 2D pose representation.”
Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks (2019, methodology): “In this research, we follow the evaluation protocols in prior work [17, 21, 26, 28, 37, 39], in which 5 subjects (S1, S5, S6, S7, S8) are used for training and 2 subjects (S9 and S11) are adopted for testing.”