Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition

By Pengfei Zhang, Cuiling Lan, Wenjun Zeng, Jianru Xue, Nanning Zheng. Computer Vision and Pattern Recognition, 2019

543 citations recorded by the source corpus.

Skeleton-based human action recognition has attracted great interest thanks to the easy accessibility of the human skeleton data. Recently, there is a trend of using very deep feedforward neural networks to model the 3D coordinates of joints without considering the computational efficiency. In this paper, we propose a simple yet effective semantics-guided neural network (SGN) for skeleton-based action recognition. We explicitly introduce the high level semantics of joints (joint type and frame index) into the network to enhance the feature representation capability. In addition, we exploit the relationship of joints hierarchically through two modules, i.e., a joint-level module for modeling the correlations of joints in the same frame and a framelevel module for modeling the dependencies of frames by taking the joints in the same frame as a whole. A strong baseline is proposed to facilitate the study of this field. With an order of magnitude smaller model size than most previous works, SGN achieves the state-of-the-art performance on the NTU60, NTU120, and SYSU datasets.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/3f057477421de2d6464690d66c61c8b3e649605a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Jun Liu
2. Hossein Rahmani
3. Qiuhong Ke
4. Liang Wang
5. Zhang Zhang
6. Caifeng Shan
7. Yisheng Song
8. Zehua Sun

In Computer Vision and Pattern Recognition

1. Jun Liu
2. Liang Wang
3. Hossein Rahmani
4. Qiuhong Ke
5. G. Wang
6. Zehua Sun
7. Zhang Zhang
8. Caifeng Shan

In Human-Computer Interaction

1. Liang Wang
2. Caifeng Shan
3. Yisheng Song
4. Zhang Zhang
5. Qiaoyong Zhong
6. Shiliang Pu
7. Di Xie
8. Jun Liu

Citations in the corpus

• Human Action Recognition From Various Data Modalities: A Review (2020, background): “Highlevel semantics of joints were introduced for HAR in [200].”
• Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition (2020, methodology): “ 78.5† 2s-AGCN [23] CVPR19 22.3† 6.94★ 88.5 95.1 82.5† 84.2† AGC-LSTM [24] CVPR19 – 22.89★ 89.2 95.0 – – DGNN [22] CVPR19 – 26.24★ 89.9 96.1 – – AS-GCN+DH-TCN [18] arXiv19 – – 85.3 92.8 78.3 79.8 SGN [35] arXiv19 188.0 1.8 86.6 93.4 – – PL-GCN [9] AAAI20 – 20.70★ 89.2 95.0 – – NAS-GCN [19] AAAI20 – ”
• InfoGCN: Representation Learning for Human Skeleton-based Action Recognition (2022, background): “Graph representation of the skeleton using bone connectiv-20186 ity (extrinsic topology) [27, 33, 44, 56, 60] has an inherent limitation: it can ignore possible joint relations, called an intrinsic topology.”
• Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition (2020, background): “Existing learning-based methods [12, 21, 26, 36] predict the dependency between two joints A ij independently, and hand-crafted functions (e.g. inner product) to map input features into dependency are employed.”
• Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition (2021, methodology): “Most GCN-based methods follow topology-shared manner, including aforementioned static methods [9, 20, 31] and dynamic methods [15, 24, 33, 34].”