Pose Recognition with Cascade Transformers

By Ke Li, Shijie Wang, Xiang Zhang, Yifan Xu, Weijian Xu, Zhuowen Tu. 2021

269 citations recorded by the source corpus.

In this paper, we present a regression-based pose recognition method using cascade Transformers. One way to categorize the existing approaches in this domain is to separate them into 1). heatmap-based and 2). regression-based. In general, heatmap-based methods achieve higher accuracy but are subject to various heuristic designs (not end-to-end mostly), whereas regression-based approaches attain relatively lower accuracy but they have less intermediate non-differentiable steps. Here we utilize the encoder-decoder structure in Transformers to perform regression-based person and keypoint detection that is general-purpose and requires less heuristic design compared with the existing approaches. We demonstrate the keypoint hypothesis (query) refinement process across different self-attention layers to reveal the recursive self-attention mechanism in Transformers. In the experiments, we report competitive results for pose recognition when compared with the competing regression-based methods.

Subfields: Computer Vision and Pattern Recognition, Model Architectures, 3D Vision & Pose, Human-Computer Interaction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Ming-Hsuan Yang
4. Yu Qiao
5. Xiaogang Wang
6. Chen Change Loy
7. Hongsheng Li
8. Yi Yang

In Model Architectures

1. Luc Van Gool
2. Hongsheng Li
3. Peng Gao
4. Guanglu Song
5. Taojiannan Yang
6. Dacheng Tao
7. Hanspeter Pfister
8. Yu Qiao

In 3D Vision & Pose

1. Luc Van Gool
2. Yuhui Yuan
3. Dacheng Tao
4. Yu Qiao
5. Mubarak Shah
6. Jing Zhang
7. Jingdong Wang
8. Radu Timofte

In Human-Computer Interaction

1. Luc Van Gool
2. Dacheng Tao
3. Yu Qiao
4. Jingdong Wang
5. Haibin Ling
6. Taojiannan Yang
7. Ju Shen
8. Hanspeter Pfister

Citations in the corpus

• EDTER: Edge Detection with Transformer (2022, methodology): “Recently, it is applied in conjunction with CNN in DETR [7] and the other variants [10, 27, 31, 58, 73].”
• MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (2022, methodology): “In high-level vision, Transformer has been widely applied in image classification [7, 19, 25, 27, 49, 65], object detection [6, 22, 23, 60, 86, 93], semantic segmentation [16, 69,76,77,90,91], human pose estimation [14,15,32,37,42, 43, 53, 79, 89], etc.”
• Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “, [18, 55, 117, 119, 148, 149, 154, 155, 173, 179, 191, 223, 234, 285]) to predict joint coordinates from images as shown in Fig.”
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction (2021, background): “66,66], segmentation [6,28,55,64,65], human pose estimation [23, 24, 56], etc.”
• RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose (2023, methodology): “PRTR [33] and PETR [51] propose end-to-end multi-person pose estimation frameworks with transformers, inspired by the pioneer in detection [8].”
• UniFormer: Unifying Convolution and Self-Attention for Visual Recognition (2022, background): “It is worth noting that HRFormer [113], PRTR [50], TokenPose [54] and TransPose [109] are sophisticatedly designed for pose estimation task.”
• ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation (2022, methodology): “For example, PRTR [23] incorporates both transformer encoders and decoders to gradually refine the locations of the estimated keypoints in a cascade manner.”
• HRFormer: High-Resolution Transformer for Dense Prediction (2021, methodology): “We compare HRFormer to the representative convolutional method such as HRNet [40] and several recent transformer methods, including PRTR [23], TransPose-H-A6 [50], and TokenPose-L/D24 [24].”