Human Pose Estimation via Convolutional Part Heatmap Regression

By Adrian Bulat, Georgios Tzimiropoulos. 2016

549 citations recorded by the source corpus.

This paper is on human pose estimation using Convolutional Neural Networks. Our main contribution is a CNN cascaded architecture specifically designed for learning part relationships and spatial context, and robustly inferring pose even for the case of severe part occlusions. To this end, we propose a detection-followed-by-regression CNN cascade. The first part of our cascade outputs part detection heatmaps and the second part performs regression on these heatmaps. The benefits of the proposed architecture are multi-fold: It guides the network where to focus in the image and effectively encodes part constraints and context. More importantly, it can effectively cope with occlusions because part detection heatmaps for occluded parts provide low confidence scores which subsequently guide the regression part of our network to rely on contextual information in order to predict the location of these parts. Additionally, we show that the proposed cascade is flexible enough to readily allow the integration of various CNN architectures for both detection and regression, including recent ones based on residual learning. Finally, we illustrate that our cascade achieves top performance on the MPII and LSP data sets. Code can be downloaded from http://www.cs.nott.ac.uk/~psxab5/.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, 3D Vision & Pose, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

In Human-Computer Interaction

Citations in the corpus

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, methodology): “Detection based methods handle pose estimation as a heat map prediction problem, where each pixel in a heat map represents the detection score of a corresponding joint [7, 18].”
Deep High-Resolution Representation Learning for Visual Recognition (2019, background): “Other works include: light upsample process [7], [24], [92], [152], possibly with dilated convolutions used in the back-bone [63], [89], [113]; light downsample and heavy up-sample processes [141], recombinator networks [55]; improving skip connections with more or complicated convolutional units…”
ConvNets and ImageNet Beyond Accuracy: Explanations, Bias Detection, Adversarial Examples and Model Criticism (2017, background): “For example, the state of the art image segmentation [15, 20] or pose estimation models [23, 6] heavily rely on pre-trained Ima-1 genet features.”
Recurrent Human Pose Estimation (2016, background): “Furthermore, combining heatmaps with large convolutional kernels and deeper models [8], [23], [25], [27], [43] improves performance – since the effective receptive fields, and consequently the context captured, is increased.”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, background): “The high-to-low process aims to generate low-resolution and high-level representations, and the low-to-high process aims to produce highresolution representations [4, 11, 22, 70, 39, 60].”
A Survey of Deep Learning-Based Object Detection (2019, methodology): “Typical 2D human pose estimation methods [284][285][286][287][288][289][290] utilize”
LCR-Net: Localization-Classification-Regression for Human Pose (2017, methodology): “CNNs have been used for full-body pose estimation both 1063-6919/17 $31.00 © 2017 IEEE DOI 10.1109/CVPR.2017.134 1216 in regression [14, 31] and classification [23] approaches.”
Deep Learning-based Human Pose Estimation: A Survey (2020, background): “Therefore, there is a recent growing interest in leveraging heatmaps to represent the joint locations and developing effective CNN architectures for HPE, e.g., References [6, 7, 11, 46, 59, 124, 129, 145, 171, 200, 231, 232, 254, 259, 269, 285].”
How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) (2017, background): “A detailed review of state-ofthe-art methods on landmark localization for human pose estimation is beyond the scope of this work, please see [39, 38, 24, 17, 27, 42, 23, 5].”
Human Pose Regression by Combining Indirect Part Detection and Contextual Information (2017, unknown): “More recent methods have shown signiﬁcant improvements on accuracy by using fully convolutional models to generate belief maps (or heat maps) for joint probabilities [34, 3, 5, 27, 13].”
Associative Embedding: End-to-End Learning for Joint Detection and Grouping (2016, background): “Multiperson Pose Estimation Recent methods have made great progress improving human pose estimation in images in particular for single person pose estimation [50, 48, 52, 40, 8, 5, 41, 4, 14, 19, 34, 26, 7, 49, 44].”
Monocular Total Capture: Posing Face, Body, and Hands in the Wild (2018, background): “Single Image 2D Human Pose Estimation: Over the last few years, great progress has been made in detecting 2D human body keypoints from a single image [64, 63, 11, 68, 38, 15] by leveraging large-scale manually annotated datasets [28, 5] with deep Convolutional Neural Network (CNN) framework.”
Toward fast and accurate human pose estimation via soft-gated skip connections (2020, methodology): “[38] Bulat&Tzimiropoulos [3] Chu et al.”
Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation (2018, background): “With recent advances in Deep Neural Networks (DNNs), image based human pose estimation has achieved significant progress in the past few years [4, 35, 15, 28, 22, 11, 17, 40, 3, 24].”
PoseTrack: A Benchmark for Human Pose Estimation and Tracking (2017, background): “Human pose estimation has recently made significant progress on the tasks of single person pose estimation in individual frames [46, 45, 44, 4, 49, 15, 18, 31, 2, 36] and videos [34, 6, 21, 12] as well as multi-person pose estimation in monocular images [35, 18, 20, 3, 32].”