Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition

By Yisheng Song, Zhang Zhang, Caifeng Shan, Liang Wang. ACM Multimedia, 2020

348 citations recorded by the source corpus.

One essential problem in skeleton-based action recognition is how to extract discriminative features over all skeleton joints. However, the complexity of the State-Of-The-Art (SOTA) models of this task tends to be exceedingly sophisticated and over-parameterized, where the low efficiency in model training and inference has obstructed the development in the field, especially for large-scale action datasets. In this work, we propose an efficient but strong baseline based on Graph Convolutional Network (GCN), where three main improvements are aggregated, i.e., early fused Multiple Input Branches (MIB), Residual GCN (ResGCN) with bottleneck structure and Part-wise Attention (PartAtt) block. Firstly, an MIB is designed to enrich informative skeleton features and remain compact representations at an early fusion stage. Then, inspired by the success of the ResNet architecture in Convolutional Neural Network (CNN), a ResGCN module is introduced in GCN to alleviate computational costs and reduce learning difficulties in model training while maintain the model accuracy. Finally, a PartAtt block is proposed to discover the most essential body parts over a whole action sequence and obtain more explainable representations for different skeleton action sequences. Extensive experiments on two large-scale datasets, i.e., NTU RGB+D 60 and 120, validate that the proposed baseline slightly outperforms other SOTA models and meanwhile requires much fewer parameters during training and inference procedures, e.g., at most 34 times less than DGNN, which is one of the best SOTA methods.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/250e25d3b30d4315e81ea2bc2569136f17136433

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dahua Lin
2. Kai Chen
3. Bo Dai
4. Zhang Zhang
5. Caifeng Shan
6. Yisheng Song
7. Liang Wang
8. Yue Zhao

In Biomedical Engineering

1. Zhang Zhang
2. Caifeng Shan
3. Yisheng Song
4. Yue Zhao
5. Bo Dai
6. Dian Shao
7. Liang Wang
8. Dahua Lin

In Computer Vision and Pattern Recognition

1. Bo Dai
2. Kai Chen
3. Liang Wang
4. Dahua Lin
5. Luc Van Gool
6. Dacheng Tao
7. Chen Change Loy
8. Xiaogang Wang

Citations in the corpus

• InfoGCN: Representation Learning for Human Skeleton-based Action Recognition (2022, neutral): “…[44] 88.5 95.1 SGN [60] 89.0 94.5 DGNN [43] 89.9 96.1 ST-TR-agcn [39] 90.3 96.3 Shift-GCN [9] 90.7 96.5 DC-GCN+ADG [8] 90.8 96.6 PA-ResGCN-B19 [47] 90.9 96.0 DDGCN [22] 91.1 97.1 Dynamic GCN [58] 91.5 96.0 MS-G3D [33] 91.5 96.2 MST-GCN [7] 91.5 96.6 CTR-GCN [6] 92.4 96.8 Ours 93.0 97.1 Acc (”
• Revisiting Skeleton-based Action Recognition (2021, background): “Graph convolutional network is widely adopted in skeleton-based action recognition [3,7,19,55,56,71].”
• Constructing Stronger and Faster Baselines for Skeleton-Based Action Recognition (2021, methodology): “This work is an extension of an earlier and preliminary version presented in [27], namely ResGCN.”
• Gaitgraph: Graph Convolutional Network for Skeleton-Based Gait Recognition (2021, methodology): “Inspired by the success of GCNs in skeleton-based action recognition [9,10], we adapted the methods to the gait recognition task.”