Adversarial PoseNet: A Structure-Aware Convolutional Network for Human Pose Estimation

By Yu Long Chen, Chunhua Shen, Xiu-Shen Wei, Lingqiao Liu. 2017

348 citations recorded by the source corpus.

For human pose estimation in monocular images, joint occlusions and overlapping upon human bodies often result in deviated pose predictions. Under these circumstances, biologically implausible pose predictions may be produced. In contrast, human vision is able to predict poses by exploiting geometric constraints of joint inter-connectivity. To address the problem by incorporating priors about the structure of human bodies, we propose a novel structure-aware convolutional network to implicitly take such priors into account during training of the deep network. Explicit learning of such constraints is typically challenging. Instead, we design discriminators to distinguish the real poses from the fake ones (such as biologically implausible ones). If the pose generator (G) generates results that the discriminator fails to distinguish from real ones, the network successfully learns the priors.,,To better capture the structure dependency of human body joints, the generator G is designed in a stacked multi-task manner to predict poses as well as occlusion heatmaps. Then, the pose and occlusion heatmaps are sent to the discriminators to predict the likelihood of the pose being real. Training of the network follows the strategy of conditional Generative Adversarial Networks (GANs). The effectiveness of the proposed network is evaluated on two widely used human pose estimation benchmark datasets. Our approach significantly outperforms the state-of-the-art methods and almost always generates plausible human pose predictions.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

FCOS: Fully Convolutional One-Stage Object Detection (2019, background): “Recently, fully convolutional networks (FCNs) [20] have achieved tremendous success in dense prediction tasks such as semantic segmentation [20, 28, 9, 19], depth estimation [17, 31], keypoint detection [3] and counting [2].”
RGB-D-based Human Motion Recognition with Deep Learning: A Survey (2017, methodology): “as DeepPose [129], Deepercut [58] and Adversarial PoseNet [15].”
Deep High-Resolution Representation Learning for Visual Recognition (2019, methodology): “In comparison to Chen et al. [23] that uses Hourglass with large parameter and computation complexity as the backbone, our scores are better except the AUC 0 .”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, neutral): “[10] 98.”
High-Resolution Representations for Labeling Pixels and Regions (2019, methodology): “At testing, each keypoint location is predicted by transforming the highest heatvalue location from 1 / 4 to the original image space and adjusting it with a quarter offset in the direction from the highest response to the second highest response [17].”
Fast Human Pose Estimation (2018, methodology): “Compared with the top-performing alternative pose approaches [32, 10], the proposed FPD approach enables much faster and more cost-effective model inference with extremely smaller model size while simultaneously reaching the same level of human pose prediction performance.”
Distribution-Aware Coordinate Representation for Human Pose Estimation (2019, background): “Representative design improvements include sequential modelling [12, 2], receptive field expansion [32], position voting [16], intermediate supervision [20, 32], pairwise relations modelling [4], tree structure modelling [8, 35, 7, 26, 29], hierarchical context learning [37], pyramid residual learni”
Structured Knowledge Distillation for Semantic Segmentation (2019, background): “The idea of adversarial learning is also adopted in pose estimation [8], encouraging the human pose estimation result not to be distinguished from the ground-truth; and semantic segmentation [27], encouraging the estimated segmentation map not to be distinguished from the groundtruth map.”
Deep Learning-based Human Pose Estimation: A Survey (2020, background): “[28] constructed a structure-aware conditional adversarial networkśAdversarial PoseNetśwhich contains an hourglass network-based pose generator and two discriminators to discriminate reasonable body poses from unreasonable ones.”
Human Pose Regression by Combining Indirect Part Detection and Contextual Information (2017, neutral): “ 85.7 81.7 89.7 Newell et al. [27] 98.2 96.3 91.2 87.1 90.1 87.4 83.6 90.9 Chu et al. [13] 98.5 96.3 91.9 88.1 90.6 88.0 85.0 91.5 Chou et al. [11] 98.2 96.8 92.2 88.0 91.3 89.1 84.9 91.8 Chen et al. [8] 98.1 96.5 92.5 88.5 90.2 89.6 86.0 91.9 Regression based methods Rogez et al. [36] – – – – – – –”
Toward fast and accurate human pose estimation via soft-gated skip connections (2020, methodology): “A similar approach is followed in [6], where a discriminator is used to discern between feasible and biologically unfeasible poses.”
Self Adversarial Training for Human Pose Estimation (2017, background): “Another type of task is about generating heatmaps of labels as in semantic segmentation [26], saliency [29] or human pose estimation [10].”
DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning (2019, methodology): “, 2019) and could be extended to use adversarial (Goodfellow et al., 2014; Chen et al., 2017) loss functions, both of which incorporate spatial dependencies within the scene rather than modelling each video frame as a set of statistically independent pixel distributions—as is the case with current m”
Monocular human pose estimation: A survey of deep learning-based methods (2020, background): “…(Wei et al., 2016), PRM (Yang et al., 2017), feed forward module (Belagiannis and Zisserman, 2017), HRNet (Sun et al., 2019), GAN (Chou et al., 2017; Chen et al., 2017; Peng et al., 2018) (3) Body structure constraint : (Tompson et al., 2014; Lifshitz et al., 2016; Yang et al., 2016; Gkioxari…”
Integral Human Pose Regression (2017, methodology): “The most widely adopted is mean square error (or L2 distance) between the predicted heat map and ground-truth heat map with a 2D Gaussian blob centered on the ground truth joint location [41, 43, 5, 29, 8, 10, 12, 4].”
FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors (2017, methodology): “Inspired by the recent success of stacked heatmap regression in human pose estimation [3, 28], we adopt the HourGlass (HG) structure to estimate facial landmark heatmaps and parsing maps in our prior estimation network.”
Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation (2018, methodology): “Recent applications of GANs in the human pose estimation include [6] and [7].”