Learning Feature Pyramids for Human Pose Estimation

By Wei Yang, Shuang Li, Wanli Ouyang, Hongsheng Li, Xiaogang Wang. 2017

525 citations recorded by the source corpus.

Articulated human pose estimation is a fundamental yet challenging task in computer vision. The difficulty is particularly pronounced in scale variations of human body parts when camera view changes or severe foreshortening happens. Although pyramid methods are widely used to handle scale changes at inference time, learning feature pyramids in deep convolutional neural networks (DCNNs) is still not well explored. In this work, we design a Pyramid Residual Module (PRMs) to enhance the invariance in scales of DCNNs. Given input features, the PRMs learn convolutional filters on various scales of input features, which are obtained with different subsampling ratios in a multibranch network. Moreover, we observe that it is inappropriate to adopt existing methods to initialize the weights of multi-branch networks, which achieve superior performance than plain networks in many tasks recently. Therefore, we provide theoretic derivation to extend the current weight initialization scheme to multi-branch network structures. We investigate our method on two standard benchmarks for human pose estimation. Our approach obtains state-of-the-art results on both benchmarks. Code is available at https://github.com/bearpaw/PyraNet.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, methodology): “[56] replaced the residual unit by a Pyramid Residual Module (PRM).”
Deep High-Resolution Representation Learning for Visual Recognition (2019, methodology): “…process (e.g., VGGNet), with skipping connection over some mirrored layers to transform the pooling indices, e.g., SegNet [3] and DeconvNet [107], or copying the feature maps, e.g., U-Net [119] and Hourglass [8], [9], [27], [31], [68], [105], [134], [163], [165], encoder-decoder [112], and so on.”
Compositional Human Pose Regression (2017, background): “Another category of methods use a multi-stage architecture (Wei et al., 2016; Newell et al., 2016; Carreira et al., 2016; Bulat and Tzimiropoulos, 2016; Gkioxari et al., 2016; Chu et al., 2017; Yang et al., 2017).”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, methodology): “Following [14, 74, 60], a six-scale pyramid testing procedure is performed.”
Deep Feature Pyramid Reconfiguration for Object Detection (2018, methodology): “Such technology is successfully used in object detection [15,31], segmentation [19], pose estimation [47,5], etc. Ideally, the pyramid features in ConvNets should: (1) reuse multi-scale features from diﬀerent layers of a single network, and (2) improve features with strong semantics at all scales.”
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields (2018, methodology): “To obtain reliable local observations of body parts, Convolutional Neural Networks (CNNs) have been widely used, and have significantly boosted the accuracy on body pose estimation [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32].”
Look at Boundary: A Boundary-Aware Face Alignment Algorithm (2018, methodology): “Following previous work in face alignment [15, 60] and human pose [35, 12, 62], we use stacked hourglass as the baseline of boundary heatmap estimator.”
Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning (2018, background): “Due to its compelling utility for HCI, motion capture, and security applications, a large body of work has since developed in this joint detection domain [53, 52, 39, 37, 62, 38, 20, 14].”
3FabRec: Fast Few-Shot Face Alignment by Reconstruction (2019, methodology): “Images were sourced from the WIDER FACE dataset [52] and were manually annotated with a much larger number of 98 landmarks.”
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks (2019, background): “In recent years, we have witnessed the remarkable achievements of CNNs in many computer vision tasks [40, 48, 37, 51, 24].”
Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition (2017, background): “Intermediate supervision has been proven to be practical training strategy in many other computer vision tasks [22, 45, 46, 24, 6].”
Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression (2021, methodology): “The convolutional neural network (CNN) solutions [17, 35, 56, 70, 43, 45, 49, 74, 54] to human pose estimation have shown superior performance over the conventional methods, such as the probabilistic graphical model or the pictorial structure model [72, 50].”
Toward fast and accurate human pose estimation via soft-gated skip connections (2020, methodology): “Current state-of-the-art on single person pose estimation is held by variants of the so-called HourGlass [23], [38], [3], [19], [8], [39], [6] and U-Net architectures [28], [33], [35].”
StarMap for Category-Agnostic Keypoint and Viewpoint Estimation (2018, background): “…Li et al. [14] extended this approach to 3D. Starting from Tompson et al. [34], the heatmap representation has dominated the 2D key-point estimation community and has achieved great success in both 2D human pose estimation [24,40,46] and single category man-made object keypoint detection [42,41].”
Simple Baselines for Human Pose Estimation and Tracking (2018, methodology): “Hourglass [4] is the dominant approach on MPII benchmark as it is the basis for all leading methods [5,6,7,8].”