Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

By Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh. 2016

7,345 citations recorded by the source corpus.

We present an approach to efficiently detect the 2D pose of multiple people in an image. The approach uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn to associate body parts with individuals in the image. The architecture encodes global context, allowing a greedy bottom-up parsing step that maintains high accuracy while achieving realtime performance, irrespective of the number of people in the image. The architecture is designed to jointly learn part locations and their association via two branches of the same sequential prediction process. Our method placed first in the inaugural COCO 2016 keypoints challenge, and significantly exceeds the previous state-of-the-art result on the MPII Multi-Person benchmark, both in performance and efficiency.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Objects as Points (2019, methodology): “We use the default output stride ofR = 4 in literature [4,40,42].”
Towards Accurate Multi-person Pose Estimation in the Wild (2017, background): “Recent work [33, 24, 8, 23] has advocated the bottom-up approach; in”
Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models (2023, neutral): “Remove Objects from Image [10, 35] Replace Objects from Image [10, 35] Change Image by the Text [35] Image Question Answering [23] Image-to-Text [23] Text-to-Image [35] Image-to-Edge [45] Edge-to-Image [53] Image-to-Line [16] Line-to-Image [53] Image-to-Hed [44] Hed-to-Image [53] Image-to-Seg [24] S”
Simitate: A Hybrid Imitation Learning Benchmark (2019, neutral): “…for reasonable accurate object recognition [1], [2], [3], detection [4], [5], semantic segmentation [6], [7], [8] and human pose estimations [9], [10], [11], we believe in advancing these fundamental approaches to actual scene understanding and even replication with a mobile domestic…”
MoCoGAN: Decomposing Motion and Content for Video Generation (2017, methodology): “For each clip, we applied a human pose estimator [4] and cropped the clip so that the performer is in the center.”
Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization (2018, methodology): “Models pretrained on Imagenet [2] have enabled the development of feature extractors achieving strong performance on a variety of related still-image analysis tasks, including object detection [3, 4], pose estimation [5, 6] and semantic segmentation [7, 8].”
Bottleneck Transformers for Visual Recognition (2021, background): “Incorporating self-attention for other computer vision tasks such as keypoint detection [9] and 3D shape prediction [23]; studying self-attention architectures for self-supervised learning in computer vision [29, 26, 59, 11, 24, 12]; and scaling to much larger datasets such as JFT, YFCC and Instagra”
Video-to-Video Synthesis (2018, methodology): “We extract human poses with DensePose [23] and OpenPose [7], and directly concatenate the results together.”
Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition (2019, background): “It is well-established [20, 32] that pre-training on large datasets followed by fine-tuning on target datasets boosts performance, especially when target datasets are small [3, 10, 26, 49].”
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes (2017, methodology): “A straightforward way for localizing the 2D object center is to directly detect the center point as in existing key point detection methods [22, 7].”
Detecting and Recognizing Human-Object Interactions (2017, background): “, detecting objects [10, 9, 27] and estimating human actions/poses [12, 32, 2], has witnessed significant improvements thanks to deep learning visual representations [18, 30, 31, 17].”
SPGNet: Semantic Prediction Guidance for Scene Parsing (2019, background): “On the other hand, the multi-stage encoder-decoder models [5, 28, 33, 41, 42, 61, 68], also known as stacked hourglass networks [42], reﬁne the keypoint estimation iteratively by propagating information across stages for the task of human pose estimation.”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, background): “This paper is interested in single-person pose estimation, which is the basis of other related problems, such as multiperson pose estimation [6, 26, 32, 38, 46, 55, 40, 45, 17, 69],”
Bottom-Up Object Detection by Grouping Extreme and Center Points (2019, methodology): “The network is trained in a fully supervised way, with either an L2 loss to a rendered Gaussian map [3,5,30,49] or with a per-pixel logistic regression loss [22, 34, 35].”