Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition

By Zhan Chen, Sicheng Li, Bing Yang, Qinghan Li, Hong Liu. AAAI Conference on Artificial Intelligence, 2021

363 citations recorded by the source corpus.

Graph convolutional networks have been widely used for skeleton-based action recognition due to their excellent modeling ability of non-Euclidean data. As the graph convolution is a local operation, it can only utilize the short-range joint dependencies and short-term trajectory but fails to directly model the distant joints relations and long-range temporal information that are vital to distinguishing various actions. To solve this problem, we present a multi-scale spatial graph convolution (MS-GC) module and a multi-scale temporal graph convolution (MT-GC) module to enrich the receptive field of the model in spatial and temporal dimensions. Concretely, the MS-GC and MT-GC modules decompose the corresponding local graph convolution into a set of sub-graph convolution, forming a hierarchical residual architecture. Without introducing additional parameters, the features will be processed with a series of sub-graph convolutions, and each node could complete multiple spatial and temporal aggregations with its neighborhoods. The final equivalent receptive field is accordingly enlarged, which is capable of capturing both short- and long-range dependencies in spatial and temporal domains. By coupling these two modules as a basic block, we further propose a multi-scale spatial temporal graph convolutional network (MST-GCN), which stacks multiple blocks to learn effective motion representations for action recognition. The proposed MST-GCN achieves remarkable performance on three challenging benchmark datasets, NTU RGB+D, NTU-120 RGB+D and Kinetics-Skeleton, for skeleton-based action recognition.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/32cf93d32f5ad013752b95b0ab7d64bcb1d3d6ff

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

Citations in the corpus

InfoGCN: Representation Learning for Human Skeleton-based Action Recognition (2022, methodology): “In particular, human action recognition based on the skeleton [6,7,19,44,58] has attracted much interest in computer vision because of its robustness against a cluttered background.”
Constructing Stronger and Faster Baselines for Skeleton-Based Action Recognition (2021, methodology): “The inferior performance of the EfficientGCN in cross-view action recognition can be explained from two aspects: 1) in contrast to the 3-stream fusion in the EfficientGCN, the DC-GCN+ADG, the 4sShift-GCN and the MST-GCN all adopt 4-stream (joint, bone, motion, and bone motion) fusion strategy, where”