Convolutional Pose Machines
By Shih-En Wei, Varun Ramakrishna, Takeo Kanade, Yaser Sheikh. 2016
2,922 citations recorded by the source corpus.
Pose Machines provide a sequential prediction framework for learning rich implicit spatial models. In this work we show a systematic design for how convolutional networks can be incorporated into the pose machine framework for learning image features and image-dependent spatial models for the task of pose estimation. The contribution of this paper is to implicitly model long-range dependencies between variables in structured prediction tasks such as articulated pose estimation. We achieve this by designing a sequential architecture composed of convolutional networks that directly operate on belief maps from previous stages, producing increasingly refined estimates for part locations, without the need for explicit graphical model-style inference. Our approach addresses the characteristic difficulty of vanishing gradients during training by providing a natural learning objective function that enforces intermediate supervision, thereby replenishing back-propagated gradients and conditioning the learning procedure. We demonstrate state-of-the-art performance and outperform competing methods on standard benchmarks including the MPII, LSP, and FLIC datasets.
Subfields: Aerospace Engineering, Artificial Intelligence, Computer Vision and Pattern Recognition, 3D Vision & Pose
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Aerospace Engineering
- Jitendra Malik
- Luc Van Gool
- Andreas Geiger
- Abhinav Kumar Gupta
- Xiaogang Wang
- Pascal Fua
- Andrea Vedaldi
- Cordelia Schmid
In Artificial Intelligence
- Trevor Darrell
- Luc Van Gool
- Bernt Schiele
- Dacheng Tao
- Yoshua Bengio
- Sergey Levine
- Quoc V. Le
- Mohit Bansal
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Jitendra Malik
- Xiaogang Wang
- Ming-Hsuan Yang
- Yi Yang
- Trevor Darrell
- Andrew Zisserman
In 3D Vision & Pose
- Michael J. Black
- Jitendra Malik
- Hanbyul Joo
- Angjoo Kanazawa
- Juergen Gall
- Christian Theobalt
- Kyoung Mu Lee
- Cordelia Schmid
Citations in the corpus
- Mask R-CNN (2017, methodology): “CMUPose+++ [6] is the 2016 competition winner that uses multi-scale testing, post-processing with CPM [39], and filtering with an object detector, adding a cumulative ∼5 points (clarified in personal communication).”
- Towards Accurate Multi-person Pose Estimation in the Wild (2017, background): “Cao et al. [8] obtained state-of-the-art results on the 2016 COCO person keypoints challenge [30] by combining a variation of the unary joint detector architecture from [44] with a part affinity field regression to enforce inter-joint consistency.”
- Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields (2016, methodology): “…be capable of iteratively refining confidence maps by learning implicit spatial relationship between different body parts in the single person case [38]; we show the that sequential prediction architectures can also iteratively refine PAFs while capturing spatial relationship across different…”
- DensePose: Dense Human Pose Estimation in the Wild (2018, methodology): “Inspired by the success of recent pose estimation models based on iterative refinement [47, 31] we experiment with cascaded architectures.”
- Stacked Hourglass Networks for Human Pose Estimation (2016, background): “Some approaches tackle this with the use of separate pipelines that process the image independently at multiple resolutions and combine features later on in the network [15,18].”
- UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory (2016, background): “…[14, 27,67], region proposal generation [30,82,85], instance segmentation [21, 37, 82], pose estimation, part segmentation, and landmark localization [3,15,42,75,81,96,98,104,111], as well as the large body of works around object detection and image classification e.g. [22,31,32,40,52,85,89,93,94].”
- 2D/3D Pose Estimation and Action Recognition Using Multitask Deep Learning (2018, background): “The problem of human pose estimation has been intensively studied in the last years, from Pictorial Structures [2, 17, 37] to more recent CNN approaches [34, 25, 38, 20, 41, 54, 5, 51, 52, 36].”
- End-to-End Recovery of Human Shape and Pose (2017, methodology): “Two stage methods first predict 2D joint locations using 2D pose detectors [30, 49, 54] or ground truth 2D pose and then predict 3D joint locations from the 2D joints either by regression [26, 29] or model fitting, where a common approach exploits a learned dictionary of 3D skeletons [2, 34, 47, 37,”
- Simitate: A Hybrid Imitation Learning Benchmark (2019, positive): “We provide human body keypoints extracted with OpenPose[11] and projected using the depth channel into world coordinates as well.”
- NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search (2018, methodology): “Elsken et al. Thomas Elsken (2018) use a simple hill climbing method along with a network morphism Wei et al. (2016b) approach to optimize network architectures quickly on limited resources.”
- Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization (2018, methodology): “Models pretrained on Imagenet [2] have enabled the development of feature extractors achieving strong performance on a variety of related still-image analysis tasks, including object detection [3, 4], pose estimation [5, 6] and semantic segmentation [7, 8].”
- DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model (2016, methodology): “[8,11] argue about the importance of large receptive fields and propose a complex hierarchical architecture predicting parts at multiple resolution levels.”
- Recurrent Human Pose Estimation (2016, background): “Furthermore, combining heatmaps with large convolutional kernels and deeper models [8], [23], [25], [27], [43] improves performance – since the effective receptive fields, and consequently the context captured, is increased.”
- Recurrent Pixel Embedding for Instance Grouping (2017, background): “…convolutional neural nets (CNNs) at image classification has spawned a flurry of work in computer vision on adapting these models to pixel-level image understanding tasks, such as boundary detection [1, 90, 64], semantic segmentation [60, 10, 46], optical flow [87, 20], and pose estimation [85, 7].”
- Detecting and Recognizing Human-Object Interactions (2017, background): “, detecting objects [10, 9, 27] and estimating human actions/poses [12, 32, 2], has witnessed significant improvements thanks to deep learning visual representations [18, 30, 31, 17].”
- SPGNet: Semantic Prediction Guidance for Scene Parsing (2019, methodology): “Multi-Stage Networks: Multi-stage networks [5,27,32,40, 41, 51, 57, 59, 66] have been widely used and explored in human pose estimation.”
- Compositional Human Pose Regression (2017, methodology): “Another category of methods use a multi-stage architecture [11, 6, 31, 5, 47, 18, 14].”
- Deep High-Resolution Representation Learning for Human Pose Estimation (2019, methodology): “The hourglass approach [39] and the convolutional pose machine approach [67] process the intermediate heatmaps as the input or a part of the input of the remaining subnetwork.”
- Survey on Emotional Body Gesture Recognition (2018, background): “tions by leveraging the fact that parts occur in consistent geometric configurations [42].”