Cascaded Pyramid Network for Multi-person Pose Estimation

By Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

1,677 citations recorded by the source corpus.

The topic of multi-person pose estimation has been largely improved recently, especially with the development of convolutional neural network. However, there still exist a lot of challenging cases, such as occluded keypoints, invisible keypoints and complex background, which cannot be well addressed. In this paper, we present a novel network structure called Cascaded Pyramid Network (CPN) which targets to relieve the problem from these "hard" keypoints. More specifically, our algorithm includes two stages: GlobalNet and RefineNet. GlobalNet is a feature pyramid network which can successfully localize the "simple" keypoints like eyes and hands but may fail to precisely recognize the occluded or invisible keypoints. Our RefineNet tries explicitly handling the "hard" keypoints by integrating all levels of feature representations from the GlobalNet together with an online hard keypoint mining loss. In general, to address the multi-person pose estimation problem, a top-down pipeline is adopted to first generate a set of human bounding boxes based on a detector, followed by our CPN for keypoint localization in each human bounding box. Based on the proposed algorithm, we achieve state-of-art results on the COCO keypoint benchmark, with average precision at 73.0 on the COCO test-dev dataset and 72.1 on the COCO test-challenge dataset, which is a 19% relative improvement compared with 60.5 from the COCO 2016 keypoint challenge. Code1 and the detection results for person used will be publicly available for further research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1deb7f96fc92d5c9e04d2cbb277473fee878e144

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

PifPaf: Composite Fields for Human Pose Estimation (2019, methodology): “Examples of top-down methods are PoseNet [35], RMPE [10], CFN [20], Mask R-CNN [18, 15] and more recently CPN [6] and MSRA [44].”
Bottom-Up Object Detection by Grouping Extreme and Center Points (2019, methodology): “The network is trained in a fully supervised way, with either an L2 loss to a rendered Gaussian map [3,5,30,49] or with a per-pixel logistic regression loss [22, 34, 35].”
Distribution-Aware Coordinate Representation for Human Pose Estimation (2019, methodology): “0 CPN [6] ResNet-Inception 384× 288 - 72.”
Deep Learning-based Human Pose Estimation: A Survey (2020, neutral): “3 2018 [90] no ResNet-Inception 384×288 72.”
Simple Baselines for Human Pose Estimation and Tracking (2018, methodology): “Note that the results of Hourglass [22] and CPN [6] are cited from [6] and not implemented by us.”
YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss (2022, background): “Top-down [30], [8], [12], [13], [19] or two-stage approaches are currently state of the art.”
Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image (2019, background): “Recently, many top-down approaches [5, 11, 41] for the 2D multi-person pose estimation have shown noticeable performance improvement.”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, methodology): “We follow the widely-adopted pipeline [39, 70, 11] to predict human keypoints using a convolutional network, which is composed of a stem consisting of two strided convolutions decreasing the resolution, a main body outputting the feature maps with the same resolution as its input feature maps, and a”
Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition (2020, background): “nterms of data size. This makes it possible to design more lightweight models. Skeleton data can be easily captured by depth cameras (e.g. Kinetics) or estimated with human pose estimation algorithms [4, 5, 31]. In the task of skeleton-based action recognition, we are given a time series of human jo”
AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time (2022, methodology): “Most top-down based methods [28], [30], [31], [65] would”
A Survey of Deep Learning-Based Object Detection (2019, background): “The best performing methods [11], [306]–[308] typically base on one-stage backbone networks.”
Rethinking Counting and Localization in Crowds: A Purely Point-Based Framework (2021, methodology): “To name a few, in the field of pose estimation, some methods adopt heatmap regression [4, 37] or direct point regression [33, 38] to predict the locations of pre-defined keypoints.”
Whole-Body Human Pose Estimation in the Wild (2020, methodology): “Top-down methods [9,12,16,32,38,42,53,65] first detect the bounding boxes and then predict the human body keypoints in each box.”
RMPE: Regional Multi-person Pose Estimation (2016, methodology): “And we believe that using the pose network from [10] can further boost our performance.”
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields (2018, methodology): “Megvii (Face++) [43] and MSRA [44] GitHub repositories do not include the person detector they use and only provide pose estimation results given a cropped person.”
Multi-Stage Progressive Image Restoration (2021, background): “In contrast, multi-stage networks are shown to be more effective than their single-stage counterparts in highlevel vision problems such as pose-estimation [14, 46, 54], scene parsing [15] and action segmentation [20, 26, 45].”
Fast Fourier Convolution (2020, neutral): “4 - CPN + OHKM [3] ResNet-50 384× 288 71.”
MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video (2022, methodology): “We use 2D keypoints from 2D pose detector [5, 36] or 2D ground truth to analyze the performance of our framework.”