DensePose: Dense Human Pose Estimation in the Wild

By Riza Alp Güler, Natalia Neverova, Iasonas Kokkinos. 2018

1,611 citations recorded by the source corpus.

In this work we establish dense correspondences between an RGB image and a surface-based representation of the human body, a task we refer to as dense human pose estimation. We gather dense correspondences for 50K persons appearing in the COCO dataset by introducing an efficient annotation pipeline. We then use our dataset to train CNN-based systems that deliver dense correspondence 'in the wild', namely in the presence of background, occlusions and scale variations. We improve our training set's effectiveness by training an inpainting network that can fill in missing ground truth values and report improvements with respect to the best results that would be achievable in the past. We experiment with fully-convolutional networks and region-based models and observe a superiority of the latter. We further improve accuracy through cascading, obtaining a system that delivers highly-accurate results at multiple frames per second on a single gpu. Supplementary materials, data, code, and videos are provided on the project page http://densepose.org.

Subfields: Computer Vision and Pattern Recognition, Endocrinology, Diabetes and Metabolism

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Endocrinology, Diabetes and Metabolism

Citations in the corpus

Momentum Contrast for Unsupervised Visual Representation Learning (2019, methodology): “COCO dense pose estimation [1]: MoCo substantially outperforms supervised pre-training, e.g ., by 3.7 points in AP dp 75 , in this highly localization-sensitive task.”
Video-to-Video Synthesis (2018, methodology): “We extract human poses with DensePose [23] and OpenPose [7], and directly concatenate the results together.”
TAP-Vid: A Benchmark for Tracking Any Point in a Video (2022, neutral): “Semantic Keypoint Tracking typically chooses a (very small) set of object categories and hand-defines a small number of keypoints to track [29, 50, 60, 64, 70, 80], though some variants track densely for humans [19, 44].”
Single Image 3D Hand Reconstruction with Mesh Convolutions (2019, methodology): “In terms of applications beyond computer graphics, our system can be modified to solve the problems of hand pose estimation [29, 30, 39], fingertip detection [35], or dense correspondence computation [13, 14].”
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization (2019, methodology): “While texture inference is often performed on either a 2D parameterization of the surface [31, 21] or in view-space [39], PIFu enables us to directly predict the RGB colors on the surface geometry by deﬁning s in Eq.”
LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching (2023, methodology): “Then, we rely on ControlNet [50] conditioned on DensePose [10] signals to offer more robust supervision.”
Survey on Deep Neural Networks in Speech and Vision Systems (2019, background): “Another work extends pose estimation to dense pose estimation [113] where the goal is to generate a 3D mesh surface of an individual from 2D images.”
Adversarial Defense Via Local Flatness Regularization (2019, neutral): “Deep neural networks (DNNs) have been widely and successfully used in many computer vision area, such as pose estimation [1, 2, 3], object detection [4, 5, 6] and super-resolution [7, 8, 9].”
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction (2020, methodology): “For example, by ﬁtting the SMPL model to the 2D keypoint detections [34] and other dense shape cues [35], the shape and pose parameters can be automatically obtained from a single image [10], [36].”
BodyNet: Volumetric Inference of 3D Human Body Shapes (2018, background): “While our results are on par with previous approaches on 2D metrics, we note that the provided manual segmentations and the 3D SMPL ﬁts [34] are noisy and aﬀect both the training and the evaluation [48].”
Deep 3D human pose estimation: A review (2021, positive): “Surface-Based Model: Recent, a new model of the human body: DensePose (Alp Güler et al., 2018) is recently proposed, considering the fact that sparse correspondence of the image and keypoints is not enough to capture the status of the human body.”
Fast animal pose estimation using deep neural networks (2018, background): “Algorithms built for human images can deal with large amounts of heterogeneity in body shape, environment and image quality, but use very large labeled training sets of images 37–39 .”
Gaze360: Physically Unconstrained Gaze Estimation in the Wild (2019, background): “In recent years, while methods for related human modeling problems such as 2D body pose and face tracking have achieved impressive success by leveraging the representational power of deep convolutional neural networks along with very large annotated datasets [2, 6, 9, 14, 26], methods for gaze estim”
A model-based gait recognition method with body pose and human prior knowledge (2020, positive): “Such as DensePose in [42] can model the human body surface with a mesh.”
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation (2019, methodology): “Such techniques have been successfully employed for body pose estimation [43, 17], camera relocalization [37, 46] and 6D object pose estimation [4].”
Single-Shot Freestyle Dance Reenactment (2020, methodology): “(iv) The DensePose [34] network and (v) the OpenPose [4] network are both used to obtain pose information from each frame, as a way to represent the input of P2B.”