Stacked Hourglass Networks for Human Pose Estimation
By Alejandro Newell, Kaiyu Yang, Jia Deng. 2016
5,506 citations recorded by the source corpus.
This work introduces a novel convolutional network architecture for the task of human pose estimation. Features are processed across all scales and consolidated to best capture the various spatial relationships associated with the body. We show how repeated bottom-up, top-down processing used in conjunction with intermediate supervision is critical to improving the performance of the network. We refer to the architecture as a “stacked hourglass” network based on the successive steps of pooling and upsampling that are done to produce a final set of predictions. State-of-the-art results are achieved on the FLIC and MPII benchmarks outcompeting all recent methods.
Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Human-Computer Interaction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Biomedical Engineering
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Jitendra Malik
- Trevor Darrell
- Yi Yang
- Andrew Zisserman
In Human-Computer Interaction
- Yichen Wei
- Gyeongsik Moon
- Kyoung Mu Lee
- Bin Xiao
- Michael J. Black
- Cordelia Schmid
- Juergen Gall
- Xiaogang Wang
Citations in the corpus
- Feature Pyramid Networks for Object Detection (2016, methodology): “…are recent methods exploiting lateral/skip connections that associate low-level feature maps across resolutions and semantic levels, including U-Net [31] and Sharp-Mask [28] for segmentation, Recombinator networks [17] for face detection, and Stacked Hourglass networks [26] for keypoint estimation.”
- Objects as Points (2019, methodology): “Hourglass The stacked Hourglass Network [30, 40] downsamples the input by 4 × , followed by two sequential hourglass modules.”
- Panoptic Feature Pyramid Networks (2019, methodology): “As an alternative to dilation, an encoder-decoder [2] or ‘U-Net’ [49] architecture can be used to increase feature resolution [26, 44, 20, 47].”
- Towards Accurate Multi-person Pose Estimation in the Wild (2017, background): “The state-of-the-art approach for single-person pose on the MPII human pose [2] and FLIC [38] datasets is the CNN model of Newell et al. [31].”
- Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields (2016, methodology): “To obtain reliable local observations of body parts, Convolu-tional Neural Networks (CNNs) have been widely used, and have significantly boosted the accuracy on body pose estimation [23, 38, 24, 34, 35, 7, 36, 6].”
- Perceiver IO: A General Architecture for Structured Inputs & Outputs (2021, background): “[67, 57]) that are in widespread use in highbandwidth domains – computer vision, multimodal processing, and elsewhere in machine learning.”
- DensePose: Dense Human Pose Estimation in the Wild (2018, methodology): “Inspired by the success of recent pose estimation models based on iterative refinement [47, 31] we experiment with cascaded architectures.”
- Learning from Simulated and Unsupervised Images through Adversarial Training (2016, methodology): “For hand pose estimation, we use the Stacked Hourglass Net of [25] 2 hourglass blocks, and an output heatmap size 64× 64.”
- Squeeze-and-Excitation Networks (2017, methodology): “More recent work has sought to better model spatial dependence [1, 31] and incorporate spatial attention [19].”
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation (2018, methodology): “Encoder-decoder: The encoder-decoder networks have been successfully applied to many computer vision tasks, including human pose estimation [53], object detection [45, 66, 19], and semantic segmentation [49, 54, 61, 3, 43, 59, 57, 33, 76, 20].”
- Dual Path Networks (2017, methodology): “In particular, we focus on analyzing and reforming the skip connection, which has been widely used in designing modern deep neural networks and offers remarkable success in many applications [16, 7, 20, 14, 5].”
- MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, methodology): “Besides this communication module, our MaX-DeepLab employs a stackedhourglass-style decoder [78, 71, 19].”
- A Point Set Generation Network for 3D Object Reconstruction from a Single Image (2016, methodology): “To pursue even better performance, we introduce the hourglass version in Fig 2 (bottom), inspired by [18].”
- Residual Attention Network for Image Classification (2017, background): “The recent stacked hourglass network [24] fuses information from multiple scales to predict human pose, and benefits from encoding both global and local information.”
- AutoGAN: Neural Architecture Search for Generative Adversarial Networks (2019, background): “In most other computer vision tasks, a lot of the progress arises from the improved design of network architectures, such as image classification [15, 16, 52, 51, 35], segmentation [8, 9, 49], and pose estimation [43].”
- NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection (2019, background): “To address this issue, recent works on human pose estimation, image segmentation, and object detection [8, 11, 22, 28, 31] introduce cross-scale connections in ConvNets that connect internal feature layers in different scales.”
- Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation (2019, methodology): “(c) Network level architecture used in Stacked Hourglass [55].”
- StairNet: Top-Down Semantic Aggregation for Accurate One Shot Detection (2017, background): “[8, 25, 26] have shown that encoder-decoder, hourglass shape, is effective to propagate the context information.”