Constructing Stronger and Faster Baselines for Skeleton-Based Action Recognition

By Yisheng Song, Zhang Zhang, Caifeng Shan, Liang Wang. 2021

427 citations recorded by the source corpus.

One essential problem in skeleton-based action recognition is how to extract discriminative features over all skeleton joints. However, the complexity of the recent State-Of-The-Art (SOTA) models for this task tends to be exceedingly sophisticated and over-parameterized. The low efficiency in model training and inference has increased the validation costs of model architectures in large-scale datasets. To address the above issue, recent advanced separable convolutional layers are embedded into an early fused Multiple Input Branches (MIB) network, constructing an efficient Graph Convolutional Network (GCN) baseline for skeleton-based action recognition. In addition, based on such the baseline, we design a compound scaling strategy to expand the model’s width and depth synchronously, and eventually obtain a family of efficient GCN baselines with high accuracies and small amounts of trainable parameters, termed EfficientGCN-Bx, where “x” denotes the scaling coefficient. On two large-scale datasets, i.e., NTU RGB+D 60 and 120, the proposed EfficientGCN-B4 baseline outperforms other SOTA methods, e.g., achieving 92.1% accuracy on the cross-subject benchmark of NTU 60 dataset, while being 5.82× smaller and 5.85× faster than MS-G3D, which is one of the SOTA methods. The source code in PyTorch version and the pretrained models are available at https://github.com/yfsong0709/EfficientGCNv1.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Simone Calderara
2. Matteo Boschini
3. Lorenzo Bonicelli
4. Jun Liu
5. Concetto Spampinato
6. Angelo Porrello
7. Hossein Rahmani
8. Qiuhong Ke

In Biomedical Engineering

1. Jun Liu
2. Qiuhong Ke
3. Hossein Rahmani
4. Zehua Sun
5. Yue Zhao
6. Bo Dai
7. Dian Shao
8. Dahua Lin

In Computer Vision and Pattern Recognition

1. Jun Liu
2. Hossein Rahmani
3. Qiuhong Ke
4. Zehua Sun
5. Bo Dai
6. Dahua Lin
7. Kai Chen
8. Dacheng Tao

Citations in the corpus

• On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning (2022, positive): “For this test, we opt for an EfficientNet-B2 [69] backbone with no pre-train.”
• Human Action Recognition From Various Data Modalities: A Review (2020, methodology): “[204] proposed a multistream GCN model, which fuses the input branches including joint positions, motion velocities, and bone features at early stage, and utilized separable convolutional layers and a compound scaling strategy to extremely reduce the redun-”
• Revisiting Skeleton-based Action Recognition (2021, background): “Graph convolutional network is widely adopted in skeleton-based action recognition [3,7,19,55,56,71].”