Structured Feature Learning for Pose Estimation

By Xiao Chu, Wanli Ouyang, Hongsheng Li, Xiaogang Wang. 2016

269 citations recorded by the source corpus.

In this paper, we propose a structured feature learning framework to reason the correlations among body joints at the feature level in human pose estimation. Different from existing approaches of modeling structures on score maps or predicted labels, feature maps preserve substantially richer descriptions of body joints. The relationships between feature maps of joints are captured with the introduced geometrical transform kernels, which can be easily implemented with a convolution layer. Features and their relationships are jointly learned in an end-to-end learning system. A bi-directional tree structured model is proposed, so that the feature channels at a body joint can well receive information from other joints. The proposed framework improves feature learning substantially. With very simple post processing, it reaches the best mean PCP on the LSP and FLIC datasets. Compared with the baseline of learning features at each joint separately with ConvNet, the mean PCP has been improved by 18% on FLIC. The code is released to the public.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Feature Boosting Network For 3D Pose Estimation (2019, methodology): “[22] introduced a structured feature learning method to reason the relationships within the body joints for 2D pose estimation.”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, methodology): “Most traditional solutions to single-person pose estimation adopt the probabilistic graphical model or the pictorial structure model [76, 49], which is recently improved by exploiting deep learning for better modeling the unary and pair-wise energies [9, 63, 44] or imitating the iterative inference ”
Integral Human Pose Regression (2017, methodology): “[11] introduce a geometrical transform kernels together with a bi-directional tree structured model in CNN framework that”
Learning Feature Pyramids for Human Pose Estimation (2017, background): “Recently, deep models have achieved state-of-the-art results in human pose estimation [3, 29, 5, 55, 39, 12, 59, 13, 7, 40].”
Factors in Finetuning Deep Model for Object Detection with Long-Tail Distribution (2016, background): “Deep learning is found to be effective in many vision tasks [38, 4, 40, 39, 21, 24, 23, 49, 19, 34, 33, 7, 48, 31].”
Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification (2016, background): “Feature representations learned by Convolutional Neural Networks have shown their effectiveness in a wide range of visual recognition tasks [6, 12, 17, 20, 27, 44].”
Fusing 2D Uncertainty and 3D Cues for Monocular Body Pose Estimation (2016, background): “With the growing availability of large datasets and the advent of Deep Learning, the emphasis has shifted towards using discriminative 2D pose regressors [10, 12, 14, 15, 23, 30, 40, 41, 42, 60, 64, 65] to extract the 2D pose and infer a 3D one from it [8, 18, 66, 69].”
CRF-CNN: Modeling Structured Information in Human Pose Estimation (2016, methodology): “Chu et al. Chu et al. (2016) learned structured features and heuristically defined a message passing scheme.”
Human Pose Regression by Combining Indirect Part Detection and Contextual Information (2017, unknown): “8 71.9 61.9 45.4 84.3 68.7 Pishchulinet al. [32] 88.7 78.9 73.2 61.8 45.0 85.1 69.2 Chen and Yuille [7] 92.7 82.9 77.0 69.2 55.4 87.8 75.0 Yanget al. [44] 96.5 88.7 81.7 78.8 66.7 83.1 81.1 Chuet al. [12] 95.4 87.6 83.2 76.9 65.2 89.6 81.1 Pishchulinet al. [33] 96.0 91.0 83.5 82.8 71.8 96.2 85.0 Reg”
Look at Boundary: A Boundary-Aware Face Alignment Algorithm (2018, methodology): “Message passing [11, 63] has shown great power in structure modeling of human joints.”
3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information (2016, background): “Xiao et al. [15] proposed an end-to-end learning system which captures the relationships among feature maps of joints.”
Distribution-Aware Coordinate Representation for Human Pose Estimation (2019, background): “…supervision (Newell, Yang, and Deng 2016; Wei et al. 2016), pairwise relations modelling (Chen and Yuille 2014), tree structure modelling (Chu et al. 2016b; Yang et al. 2016; Chu et al. 2016a; Sun et al. 2017; Tang, Yu, and Wu 2018), pyramid residual learning (Yang et al. 2017),…”
Multi-context Attention for Human Pose Estimation (2017, background): “Significant improvements have been achieved by Convolutional Neural Networks (ConvNets) [37, 38, 9, 39, 36, 28].”
Deep Learning-based Human Pose Estimation: A Survey (2020, background): “A structured feature-level learning framework was proposed in [41] for reasoning the correlations among human body joints in HPE, which captures richer information of human body joints and improves the learning results.”
Dynamic Kernel Distillation for Efficient Pose Estimation in Videos (2019, unknown): “Thereby, we follow the conventions of still-image based pose estimation [17, 7, 31, 28], and also adopt another metric that takes torso size as reference distance.”
Through-Wall Human Pose Estimation Using Radio Signals (2018, background): “It has recently witnessed major advances and significant performance improvements [30, 27, 28, 46, 31, 20, 10, 16, 33, 12, 47, 37, 45, 13].”
End-to-End Deep Learning for Person Search (2016, background): “oblem is challenging because of complex variations of human poses, camera viewpoints, lighting, occlusion, resolution, background clutter, etc., and thus draws much research attention in recent years [4,19,21,24,35,43]. Although numerous person re-id datasets and methods have been proposed, there is”
Object Detection from Video Tubelets with Convolutional Neural Networks (2016, methodology): “Deep learning has been widely applied to various computer vision tasks such as image classification [17, 29, 30], object detection [10, 9, 28, 30, 23, 24], semantic segmentation [21, 16], human pose estimation [33, 32, 37, 4], etc.”
Multi-Scale Structure-Aware Network for Human Pose Estimation (2018, methodology): “the MPII pose benchmark [17], many methods [11], [12], [13], [14], [15] rely on repeating their pose estimation pipeline multiple times in various scales, in order to improve performance by a small margin using averaging of results.”