Human Pose Estimation Using Deep Consensus Voting

By Ita Lifshitz, Ethan Fetaya, Shimon Ullman. 2016

169 citations recorded by the source corpus.

In this paper we consider the problem of human pose estimation from a single still image. We propose a novel approach where each location in the image votes for the position of each keypoint using a convolutional neural net. The voting scheme allows us to utilize information from the whole image, rather than rely on a sparse set of keypoint locations. Using dense, multi-target votes, not only produces good keypoint predictions, but also enables us to compute image-dependent joint keypoint probabilities by looking at consensus voting. This differs from most previous methods where joint probabilities are learned from relative keypoint locations and are independent of the image. We finally combine the keypoints votes and joint probabilities in order to identify the optimal pose configuration. We show our competitive performance on the MPII Human Pose and Leeds Sports Pose datasets.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, methodology): “. 2.1. Human pose estimation 2D pose estimation. The problem of human pose estimation has been intensively studied in the last years, from Pictorial Structures [2,17,37] to more recent CNN approaches [34,25,38,20,41,54,5,51,52,36]. From the literature, we can see that there are two distinct families”
Deep High-Resolution Representation Learning for Visual Recognition (2019, background): “Other works include: light upsample process [7], [24], [92], [152], possibly with dilated convolutions used in the back-bone [63], [89], [113]; light downsample and heavy up-sample processes [141], recombinator networks [55]; improving skip connections with more or complicated convolutional units…”
Recurrent Human Pose Estimation (2016, background): “Furthermore, combining heatmaps with large convolutional kernels and deeper models [8], [23], [25], [27], [43] improves performance – since the effective receptive fields, and consequently the context captured, is increased.”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, methodology): “In [26, 50, 34], dilated convolutions are adopted in the last two stages in the ResNet or VGGNet to eliminate the spatial resolution loss, which is followed by a light lowto-high process to further increase the resolution, avoiding expensive computation cost for only using dilated convolutions [11, ”
Distribution-Aware Coordinate Representation for Human Pose Estimation (2019, background): “Representative design improvements include sequential modelling [12, 2], receptive field expansion [32], position voting [16], intermediate supervision [20, 32], pairwise relations modelling [4], tree structure modelling [8, 35, 7, 26, 29], hierarchical context learning [37], pyramid residual learni”
Human Pose Regression by Combining Indirect Part Detection and Contextual Information (2017, unknown): “78.5 71.8 72.2 61.8 83.9 74.8 Raﬁet al. [34] 97.6 87.3 80.2 76.8 66.2 93.3 81.2 Belag. and Ziss. [4] 96.0 86.7 82.2 79.4 69.4 89.4 82.1 Yuet al. [46] 98.0 93.1 88.1 82.9 72.6 83.0 85.4 Lifshitzet al. [26] 97.3 88.8 84.4 80.6 71.4 94.8 84.3 Pishchulinet al. [33] 97.0 88.8 82.0 82.4 71.8 95.8 84.3 Ins”
TokenPose: Learning Keypoint Tokens for Human Pose Estimation (2021, methodology): “Deep convolutional neural networks have been applied to human pose estimation which greatly boost the model performance [32, 13, 29, 38, 22, 17, 21, 4, 7].”
Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression (2021, methodology): “The convolutional neural network (CNN) solutions [17, 35, 56, 70, 43, 45, 49, 74, 54] to human pose estimation have shown superior performance over the conventional methods, such as the probabilistic graphical model or the pictorial structure model [72, 50].”
Adversarial PoseNet: A Structure-Aware Convolutional Network for Human Pose Estimation (2017, background): “The imageconditional models have tackled inpainting [21], image prediction from a normal map [33], future frame prediction [18], future state prediction [40], product photo generation [38], and style transfer [17].”
Knowledge-Guided Deep Fractal Neural Networks for Human Pose Estimation (2017, unknown): “, ECCV’16 [69] 97.”
You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions (2019, background): “While this makes state-of-the-art third-person pose methods poorly suited [63, 45, 58, 34, 24, 10], recent work suggests that an ego-video stream nonetheless offers implicit cues for first-person body pose [26, 69, 70].”
VNect (2017, methodology): “Learning-based discriminative methods, in particular deep learning methods [Insafutdinov et al. 2016; Lifshitz et al. 2016; Newell et al. 2016; Tompson et al. 2014], represent the current state of the art in 2D pose estimation, with some of these methods demonstrating real-time performance [Cao”
Multi-context Attention for Human Pose Estimation (2017, methodology): “Leeds Sports Pose We train our model by adding the MPII training set to the extended LSP training set with personcentric annotations, which is a standard routine [39, 22, 31, 27, 4].”
Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “[55] proposed a CNN-based method for predicting the locations of joints.”
Toward fast and accurate human pose estimation via soft-gated skip connections (2020, methodology): “[21] proposes to localize the landmarks using a dense voting technique where joint probabilities are learned from relative keypoint locations.”