Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition

By Fanfan Ye, Shiliang Pu, Qiaoyong Zhong, Di Xie, Huiming Tang. ACM Multimedia, 2020

348 citations recorded by the source corpus.

raph Convolutional Networks (GCNs) have attracted increasing interests for the task of skeleton-based action recognition. The key lies in the design of the graph structure, which encodes skeleton topology information. In this paper, we propose Dynamic GCN, in which a novel convolutional neural network named Context-encoding Network (CeN) is introduced to learn skeleton topology automatically. In particular, when learning the dependency between two joints, contextual features from the rest joints are incorporated in a global manner. CeN is extremely lightweight yet effective, and can be embedded into a graph convolutional layer. By stacking multiple CeN-enabled graph convolutional layers, we build Dynamic GCN. Notably, as a merit of CeN, dynamic graph topologies are constructed for different input samples as well as graph convolutional layers of various depths. Besides, three alternative context modeling architectures are well explored, which may serve as a guideline for future research on graph topology learning. CeN brings only ~7% extra FLOPs for the baseline model, and Dynamic GCN achieves better performance with 2x ~4x fewer FLOPs than existing methods. By further combining static physical body connections and motion modalities, we achieve state-of-the-art performance on three large-scale benchmarks, namely NTU-RGB+D, NTU-RGB+D 120 and Skeleton-Kinetics.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Endocrinology, Diabetes and Metabolism

View Paper: https://www.semanticscholar.org/paper/f41fddeeff940fd3703c75421dc7c36320376de6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computer Vision and Pattern Recognition

In Endocrinology, Diabetes and Metabolism

Liang Wang

Citations in the corpus

InfoGCN: Representation Learning for Human Skeleton-based Action Recognition (2022, background): “In particular, human action recognition based on the skeleton [6,7,19,44,58] has attracted much interest in computer vision because of its robustness against a cluttered background.”
Constructing Stronger and Faster Baselines for Skeleton-Based Action Recognition (2021, methodology): “The inferior performance of the EfficientGCN in cross-view action recognition can be explained from two aspects: 1) in contrast to the 3-stream fusion in the EfficientGCN, the DC-GCN+ADG, the 4sShift-GCN and the MST-GCN all adopt 4-stream (joint, bone, motion, and bone motion) fusion strategy, where”
Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition (2021, methodology): “[33] proposed a Dynamic GCN, where contextual features of all joints are incorporated to learn correlations between any pairs of joints.”