Recurrent Human Pose Estimation

By Vasileios Belagiannis, Andrew Zisserman. 2016

323 citations recorded by the source corpus.

We propose a ConvNet model for predicting 2D human body poses in an image. The model regresses a heatmap representation for each body keypoint, and is able to learn and represent both the part appearances and the context of the part configuration. We make the following three contributions: (i) an architecture combining a feed forward module with a recurrent module, where the recurrent module can be run iteratively to improve the performance; (ii) the model can be trained end-to-end and from scratch, with auxiliary losses incorporated to improve performance; (iii) we investigate whether keypoint visibility can also be predicted. The model is evaluated on two benchmark datasets. The result is a simple architecture that achieves performance on par with the state of the art, but without the complexity of a graphical model stage (or layers).

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Towards Accurate Multi-person Pose Estimation in the Wild (2017, methodology): “Similarly, Belagiannis & Zisserman [6] also propose a cascaded architecture to infer pairwise joint (or part) locations, which is then used to iteratively reﬁne unary joint predictions, where unlike [7], they propose iterative reﬁnement using a recursive neural network.”
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields (2016, background): “These top-down approaches directly leverage existing techniques for single-person pose estimation [17, 31, 18, 28, 29, 7, 30, 5, 6, 20], but suffer from early commitment: if the person detector fails–as it is prone to do when people are in close proximity–there is no recourse to recovery.”
UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory (2016, background): “…[14, 27,67], region proposal generation [30,82,85], instance segmentation [21, 37, 82], pose estimation, part segmentation, and landmark localization [3,15,42,75,81,96,98,104,111], as well as the large body of works around object detection and image classiﬁcation e.g. [22,31,32,40,52,85,89,93,94].”
2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, methodology): “5 Detection methods Recurrent VGG [6] 2016 61.”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, background): “Intermediate supervision or deep supervision, early developed for image classiﬁcation [34, 61], is also adopted for helping deep networks training and improving the heatmap estimation quality, e.g., [69, 40, 64, 3, 11].”
TransPose: Keypoint Localization via Transformer (2020, unknown): “Belagiannis & Zisserman, FG’17 [4] 248×248 COCO+MPII† 88.”
Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation (2018, unknown): “[2] 97.”
Deep Learning-based Human Pose Estimation: A Survey (2020, background): “, [53] [54] [39] [55] [56] [38] [40] [57] [58] [59] [60] [61] [62] [63] [64].”
Associative Embedding: End-to-End Learning for Joint Detection and Grouping (2016, background): “Multiperson Pose Estimation Recent methods have made great progress improving human pose estimation in images in particular for single person pose estimation [50, 48, 52, 40, 8, 5, 41, 4, 14, 19, 34, 26, 7, 49, 44].”
VNect (2017, unknown): “CCS Concepts: • Computing methodologies → Reconstruction ; Computational photography ; Image-based rendering ;”
A Comprehensive Analysis of Deep Regression (2018, unknown): “A cascade regression strategy is also often employ to enforce consistency among predicted joint locations [79], [81], [83], [84], [85].”
Integrated perception with recurrent multi-task neural networks (2016, background): “Typically, researchers incorporate auxiliary tasks into their target tasks, jointly train them in parallel and achieve performance gains in object tracking [30], object detection [11], facial landmark detection [31], human pose-estimation [2].”
Self Adversarial Training for Human Pose Estimation (2017, unknown): “CV’16 97.8 93.3 85.7 80.4 85.3 76.6 70.2 85.0 Gkioxary et al. [14], ECCV’16 96.2 93.1 86.7 82.1 85.2 81.4 74.1 86.1 Raﬁ et al. [34], BMVC’16 97.2 93.9 86.4 81.3 86.8 80.6 73.4 86.3 Belagiannis et al. [4], FG’17 97.7 95.0 88.2 83.0 87.9 82.6 78.4 88.1 Insafutdinov et al. [19], ECCV’16 96.8 95.2 89.3 ”
KETO: Learning Keypoint Representations for Tool Manipulation (2019, background): “Keypoint-based methods have been extensively studied in facial recognition [6], [36], [35], human pose estimation [5], [9], [40], [51], [8], [53] and tracking [39], [20], [43], [21], [10] in computer vision and pattern recognition applications.”
Cascaded Pyramid Network for Multi-person Pose Estimation (2017, methodology): “[2] apply RNN-like architectures to sequentially refine the results.”
Toward fast and accurate human pose estimation via soft-gated skip connections (2020, background): “In [2], the authors introduce a hybrid architecture that combines a normal feed-forward model with a recurrent block.”
Fast Human Pose Estimation (2018, unknown): “Belagiannis&Zisserman, FG’17[4] 97.”
TokenPose: Learning Keypoint Tokens for Human Pose Estimation (2021, background): “Intermediate supervision is widely-used to help model training and improve the heatmap estimation quality especially when networks become very deep [22, 37, 33, 2].”