TokenPose: Learning Keypoint Tokens for Human Pose Estimation

By Yanjie Li, Shoukui Zhang, Zhicheng Wang, Sen Yang, Wankou Yang, Shutao Xia, Erjin Zhou. IEEE International Conference on Computer Vision, 2021

388 citations recorded by the source corpus.

Human pose estimation deeply relies on visual clues and anatomical constraints between parts to locate keypoints. Most existing CNN-based methods do well in visual representation, however, lacking in the ability to explicitly learn the constraint relationships between keypoints. In this paper, we propose a novel approach based on Token representation for human Pose estimation (TokenPose). In detail, each keypoint is explicitly embedded as a token to simultaneously learn constraint relationships and appearance cues from images. Extensive experiments show that the small and large TokenPose models are on par with state-of-the-art CNN-based counterparts while being more lightweight. Specifically, our TokenPose-S and TokenPose-L achieve 72.5 AP and 75.8 AP on COCO validation dataset respectively, with significant reduction in parameters (↓80.6% ; ↓ 56.8%) and GFLOPs (↓ 75.3%; ↓24.7%). Code is publicly available1.

Subfields: Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/bbe78443eb2d97511d84cf90dbcc917b8e666f81

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Human-Computer Interaction

Citations in the corpus

Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “[126] built a pure transformer-based model named TokenPose to capture the constraint cues and visual appearance relationships by using token representation.”
RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose (2023, methodology): “Transformer-based architectures [55] ported from modern Natural Language Processing (NLP) have achieved great success in various vision tasks like representation learning [15, 39], object detection [8, 34, 67], semantic segmentation [66], video understanding [4, 17, 40], as well as pose estimation [”
ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation (2022, background): “TokenPose [28] and TransPose [46], instead, adopt an encoder-only transformer structure to process the features extracted by CNNs.”
A Survey on Vision Transformer (2020, methodology): “[133] proposed a novel approach based on Token representation for human Pose estimation (TokenPose).”
Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction (2021, background): “66,66], segmentation [6,28,55,64,65], human pose estimation [23, 24, 56], etc.”
MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (2022, methodology): “In high-level vision, Transformer has been widely applied in image classification [7, 19, 25, 27, 49, 65], object detection [6, 22, 23, 60, 86, 93], semantic segmentation [16, 69,76,77,90,91], human pose estimation [14,15,32,37,42, 43, 53, 79, 89], etc.”
HRFormer: High-Resolution Transformer for Dense Prediction (2021, methodology): “We compare HRFormer to the representative convolutional method such as HRNet [40] and several recent transformer methods, including PRTR [23], TransPose-H-A6 [50], and TokenPose-L/D24 [24].”