Semantic Graph Convolutional Networks for 3D Human Pose Regression

By Long Zhao, Xi Peng, Yu Tian, Mubbasir Kapadia, Dimitris N. Metaxas. Computer Vision and Pattern Recognition, 2019

621 citations recorded by the source corpus.

In this paper, we study the problem of learning Graph Convolutional Networks (GCNs) for regression. Current architectures of GCNs are limited to the small receptive field of convolution filters and shared transformation matrix for each node. To address these limitations, we propose Semantic Graph Convolutional Networks (SemGCN), a novel neural network architecture that operates on regression tasks with graph-structured data. SemGCN learns to capture semantic information such as local and global node relationships, which is not explicitly represented in the graph. These semantic relationships can be learned through end-to-end training from the ground truth without additional supervision or hand-crafted rules. We further investigate applying SemGCN to 3D human pose regression. Our formulation is intuitive and sufficient since both 2D and 3D human poses can be represented as a structured graph encoding the relationships between joints in the skeleton of a human body. We carry out comprehensive studies to validate our method. The results prove that SemGCN outperforms state of the art while using 90% fewer parameters.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/27ff37bd17a5e59149e645184ed4a0c2fb9cf2f2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Ling Shao
2. Junsong Yuan
3. Long Zhao
4. Kyoung Mu Lee
5. Gyeongsik Moon
6. Xi Peng
7. Qi Tian
8. Mubarak Shah

In Computer Vision and Pattern Recognition

1. Ling Shao
2. Luc Van Gool
3. Dacheng Tao
4. Ming-Hsuan Yang
5. Yi Yang
6. Xiaogang Wang
7. Qi Tian
8. Jitendra Malik

Citations in the corpus

• Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose (2020, methodology): “[66] developed a semantic GraphCNN to use spatial relationships between joint coordinates.”
• Simple and Deep Graph Convolutional Networks (2020, background): “Copyright 2020 by the author(s). puter vision (Zhao et al., 2019; Ma et al., 2019).”
• Learning to Learn Single Domain Generalization (2020, background): “Recent years have witnessed rapid deployment of machine learning models for broad applications [17, 42, 3, 60].”
• BernNet: Learning Arbitrary Graph Spectral Filters via Bernstein Approximation (2021, background): “Graph neural networks (GNNs) have received extensive attention from researchers due to their excellent performance on various graph learning tasks such as social analysis [24, 17, 29], drug discovery [12, 25], traffic forecasting [18, 3, 6], recommendation system [38, 32] and computer vision [39, 4]”
• Deep 3D human pose estimation: A review (2021, positive): “…the locally connected network (LCN) is proposed (Ci et al., 2019), which uses different filters for feature extraction rather than a shared filter as in GCN. Importantly, they also note the limitations of the natural skeleton adjacency matrix and make it learnable, similar to Zhao et al. (2019).”
• Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction (2020, methodology): “Several graph-based models have been employed for skeleton-based action recognition [46, 27, 34], motion prediction [31] and 3D pose estimation [47]; Different from any previous works, our model considers multiscale graphs and corresponding operations.”
• Human Pose Regression with Residual Log-likelihood Estimation (2021, methodology): “Two-stage methods [32, 13, 47, 68, 60, 9, 30, 67] lift the 2D poses to 3D space by regression.”
• Deep Learning-based Human Pose Estimation: A Survey (2020, methodology): “[297] also tackled the limitation of the shared weight matrix of convolution ilters for all the nodes in GCN.”
• Skeleton-Based Action Recognition With Multi-Stream Adaptive Graph Convolutional Networks (2019, background): “[43] propose a SemGCN to learn channel-wise weights for edges as priors implied in the graph.”
• Decoupling GCN with DropGraph Module for Skeleton-Based Action Recognition (2020, background): “DC-GCN is different from SemGCN [49] in many aspects: 1) SemGCN focus on pose regression, while we focus on action recognition.”
• Knowledge As Priors: Cross-Modal Knowledge Generalization for Datasets Without Superior Knowledge (2020, background): “Leveraging multi-modal knowledge to boost the performance of classic computer vision problems, such as classification [28, 35, 49], object detection [14, 39, 50] and gesture recognition [1, 7, 40, 44, 53, 57, 58], has emerged as a promising research field in recent years.”
• 3D Human Pose Estimation with Spatial and Temporal Transformers (2021, background): “[45] tackled a limitation of Graph Convolutional Network [19] (GCN) operations, specifically how the weight matrix is shared across nodes.”
• MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video (2022, methodology): “Benefiting from the reliable effort of 2D keypoint detection works [5,9,29,36], the recent 2Dto-3D lifting methods [6,23,25,26,43,49,51] outperformed end-to-end approaches.”