Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction

By Maosen Li, Siheng Chen, Yangheng Zhao, Ya Zhang, Yanfeng Wang, Qi Tian. Computer Vision and Pattern Recognition, 2020

365 citations recorded by the source corpus.

We propose novel dynamic multiscale graph neural networks (DMGNN) to predict 3D skeleton-based human motions. The core idea of DMGNN is to use a multiscale graph to comprehensively model the internal relations of a human body for motion feature learning. This multiscale graph is adaptive during training and dynamic across network layers. Based on this graph, we propose a multiscale graph computational unit (MGCU) to extract features at individual scales and fuse features across scales. The entire model is action-category-agnostic and follows an encoder-decoder framework. The encoder consists of a sequence of MGCUs to learn motion features. The decoder uses a proposed graph-based gate recurrent unit to generate future poses. Extensive experiments show that the proposed DMGNN outperforms state-of-the-art methods in both short and long-term predictions on the datasets of Human 3.6M and CMU Mocap. We further investigate the learned multiscale graphs for the interpretability. The codes could be downloaded from https://github.com/limaosen0/DMGNN.

Subfields: Biomedical Engineering, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/1478c3c0225368419f68aabc6b67033531d3b4c1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Zong-Yi Li
2. Kaushik Bhattacharya
3. Anima Anandkumar
4. Burigede Liu
5. Kamyar Azizzadenesheli
6. Andrew M. Stuart
7. Nikola B. Kovachki
8. Chad M. Landis

In Computer Vision and Pattern Recognition

1. Yong Li
2. Chen Gao
3. Depeng Jin
4. Yu Zheng
5. Zong-Yi Li
6. Kaushik Bhattacharya
7. Anima Anandkumar
8. Burigede Liu

In Control and Systems Engineering

1. Anima Anandkumar
2. Kamyar Azizzadenesheli
3. Andrew M. Stuart

Citations in the corpus

• Multipole Graph Neural Operator for Parametric Partial Differential Equations (2020, background): “Works on multi-resolution graphs with a U-net like structure have also recently began to emerge and seen success in imaging, classification, and semi-supervised learning [39, 1, 2, 20, 30, 34, 31].”
• A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions (2021, background): “Recently, dynamic graph neural networks [93, 123] have attracted attention, which deploys embedding propagation operations on dynamically constructed graphs.”