Revisiting Skeleton-based Action Recognition

By Haodong Duan, Yue Zhao, Kai Chen, Dian Shao, Dahua Lin, Bo Dai. Computer Vision and Pattern Recognition, 2021

724 citations recorded by the source corpus.

Human skeleton, as a compact representation of human action, has received increasing attention in recent years. Many skeleton-based action recognition methods adopt GCNs to extract features on top of human skeletons. Despite the positive results shown in these attempts, GCN-based methods are subject to limitations in robustness, interoperability, and scalability. In this work, we propose PoseConv3D, a new approach to skeleton-based action recognition. PoseConv3D relies on a 3D heatmap volume instead of a graph sequence as the base representation of human skeletons. Compared to GCN-based methods, PoseConv3D is more effective in learning spatiotemporal features, more robust against pose estimation noises, and generalizes better in cross-dataset settings. Also, PoseConv3D can handle multiple-person scenarios without additional computation costs. The hierarchical features can be easily integrated with other modalities at early fusion stages, providing a great design space to boost the performance. PoseConv3D achieves the state-of-the-art on five of six standard skeleton-based action recognition benchmarks. Once fused with other modalities, it achieves the state-of-the-art on all eight multi-modality action recognition benchmarks. Code has been made available at: https://github.com/kennymckormick/pyskl.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/f8c4c5cc82f6270b62d5f68940e444b55ea2f13c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jun Liu
2. Qiuhong Ke
3. Hossein Rahmani
4. Zehua Sun
5. Libin Liu
6. Wenjie Zhu
7. Wayne Wu
8. Xiaoxuan Ma

In Biomedical Engineering

1. Jun Liu
2. Qiuhong Ke
3. Hossein Rahmani
4. Zehua Sun
5. Wayne Wu
6. Zhaoyang Liu
7. G. Wang
8. Amir Shahroudy

In Computer Vision and Pattern Recognition

1. Qiuhong Ke
2. Jun Liu
3. Hossein Rahmani
4. Zehua Sun
5. Wayne Wu
6. Zhaoyang Liu
7. Wenjie Zhu
8. Libin Liu

Citations in the corpus

• MotionBERT: A Unified Perspective on Learning Human Motion Representations (2022, methodology): “Existing studies have shown the effectiveness of using 2D skeleton sequences for different downstream tasks [24,30,83,102].”
• Human Action Recognition From Various Data Modalities: A Review (2020, background): “PoseC3D [321] represents skeleton data as a 3D heatmap volume and passes the RGB video and the heatmap volume through a two-stream 3D CNN framework whose architecture is inspired by [112].”