Whole-Body Human Pose Estimation in the Wild

By Sheng Jin, Lumin Xu, Jin Xu, Can Wang, Wentao Liu, Chen Qian, Wanli Ouyang, Ping Luo. European Conference on Computer Vision, 2020

351 citations recorded by the source corpus.

This paper investigates the task of 2D human whole-body pose estimation, which aims to localize dense landmarks on the entire human body including face, hands, body, and feet. As existing datasets do not have whole-body annotations, previous methods have to assemble different deep models trained independently on different datasets of the human face, hand, and body, struggling with dataset biases and large model complexity. To fill in this blank, we introduce COCO-WholeBody which extends COCO dataset with whole-body annotations. To our best knowledge, it is the first benchmark that has manual annotations on the entire human body, including 133 dense landmarks with 68 on the face, 42 on hands and 23 on the body and feet. A single-network model, named ZoomNet, is devised to take into account the hierarchical structure of the full human body to solve the scale variation of different body parts of the same person. ZoomNet is able to significantly outperform existing methods on the proposed COCO-WholeBody dataset. Extensive experiments show that COCO-WholeBody not only can be used to train deep models from scratch for whole-body pose estimation but also can serve as a powerful pre-training dataset for many different tasks such as facial landmark detection and hand keypoint estimation. The dataset is publicly available at this https URL.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/9930e08d9ee3f9e5a4dce8220c5128012b594373

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Effective Whole-body Pose Estimation with Two-stages Distillation (2023, methodology): “Results of Whole-body pose estimation on COCO-WholeBody [19, 48] V1.”
TAP-Vid: A Benchmark for Tracking Any Point in a Video (2022, neutral): “Dataset Type #Videos (#Images) Duration/ Time-scale Longterm? Point Precise? Class Agnostic? Nonrigid? KITTI [16] Optical flow 156 400 frame pairs 7 3 3 3 COCO-DensePose [19] Surface Points (50k) — 7 3 7 3 COCO-WholeBody [32] Semantic Keypoints (200k) — 7 3 7 3 DAVIS [53] Masks (multi-object) 150 25”
AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time (2022, methodology): “In (a) and (b), the left definition is commonly used in 2D annotated dataset like [18], [61], [72], while the right definition is used in 3D face alignment task like [84].”
Reconstructing Hands in 3D with Transformers (2023, methodology): “Similar to [25], we initialize the annotation procedure with an existing keypoint detection model [11] to get rough keypoint locations.”
Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “[90] proposed COCO-WholeBody Dataset with whole-body annotations for HPE.”