PifPaf: Composite Fields for Human Pose Estimation

By Sven Kreiss, Lorenzo Bertoni, Alexandre Alahi. Computer Vision and Pattern Recognition, 2019

482 citations recorded by the source corpus.

We propose a new bottom-up method for multi-person 2D human pose estimation that is particularly well suited for urban mobility such as self-driving cars and delivery robots. The new method, PifPaf, uses a Part Intensity Field (PIF) to localize body parts and a Part Association Field (PAF) to associate body parts with each other to form full human poses. Our method outperforms previous methods at low resolution and in crowded, cluttered and occluded scenes thanks to (i) our new composite field PAF encoding fine-grained information and (ii) the choice of Laplace loss for regressions which incorporates a notion of uncertainty. Our architecture is based on a fully convolutional, single-shot, box-free design. We perform on par with the existing state-of-the-art bottom-up method on the standard COCO keypoint task and produce state-of-the-art results on a modified COCO keypoint task for the transportation domain.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/52b377e589220708c4ec8c4b55dda689d1bb00c4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss (2022, background): “In contrast, bottom-up [4], [5], [14], [23], [25] approaches offer constant run-time since they rely on heatmaps to detect all the keypoints in a single-shot followed by complex post-processing to group them into individual persons.”
HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation (2019, methodology): “Previous bottom-up methods [3, 30, 33, 22] mainly focus on grouping keypoints and simply use a single resolution of feature map that is 1/4 of the input image resolution to predict the heatmap of keypoints.”
RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose (2023, background): “Bottom-up algorithms [7, 10, 19, 27, 29, 41, 44, 45] detect instance-agnostic keypoints in an image and partition these keypoints to obtain the human pose.”
Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “A line of works focus on designing and improving the modules in HPE networks, e.g., References [14, 71, 72, 121, 134, 166, 180, 221, 246, 259, 283].”
AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time (2022, background): “Representative works [10], [11], [12], [13], [21], [22], [23] are reviewed.”
Multi-animal pose estimation, identification and tracking with DeepLabCut (2022, background): “To associate detected keypoints to particular individuals (assembly) several solutions have been proposed, such as part affinity fields (PAFs) 9 , associative embeddings 10,11 , transformers 12 and other mechanisms 13,14 .”
Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression (2021, methodology): “6 gain over PifPaf [31] whose GFLOPs are more than twice as many as ours, and 1 .”
BlazePose: On-device Real-time Body Pose tracking (2020, background): “Recent work [10][7] has shown significant progress on pose estimation.”
Monocular human pose estimation: A survey of deep learning-based methods (2020, methodology): “Similar to OpenPose, Kreiss et al. (2019) designed a PifPaf net to predict a Part Intensity Field (PIF) and a Part Association Field (PAF) to represent body joint locations”
Pose Recognition with Cascade Transformers (2021, methodology): “2 PifPaf [17] ResNet-101-dilation − − − 66.”