Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose

By Hongsuk Choi, Gyeongsik Moon, Kyoung Mu Lee. European Conference on Computer Vision, 2020

479 citations recorded by the source corpus.

Most of the recent deep learning-based 3D human pose and mesh estimation methods regress the pose and shape parameters of human mesh models, such as SMPL and MANO, from an input image. The first weakness of these methods is an appearance domain gap problem, due to different image appearance between train data from controlled environments, such as a laboratory, and test data from in-the-wild environments. The second weakness is that the estimation of the pose parameters is quite challenging owing to the representation issues of 3D rotations. To overcome the above weaknesses, we propose Pose2Mesh, a novel graph convolutional neural network (GraphCNN)-based system that estimates the 3D coordinates of human mesh vertices directly from the 2D human pose. The 2D human pose as input provides essential human body articulation information, while having a relatively homogeneous geometric property between the two domains. Also, the proposed system avoids the representation issues, while fully exploiting the mesh topology using a GraphCNN in a coarse-to-fine manner. We show that our Pose2Mesh outperforms the previous 3D human pose and mesh estimation methods on various benchmark datasets. The codes are publicly available this https URL.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/4f632a308c2b4a5fdb2c553219f2fe3b92a6b301

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep 3D human pose estimation: A review (2021, unknown): “Because of the small amount of information in a single image, most works adopt a parametric model like SMPL and learn to predict the shape and pose coefficients, such as (Arnab et al., 2019b; Sun et al., 2019; Sengupta et al., 2020; Choutas et al., 2020; Kolotouros et al., 2019b; Moon and Lee, 2020;”
End-to-End Human Pose and Mesh Reconstruction with Transformers (2020, methodology): “Moreover, recent proposed Pose2Mesh [8] is a cascaded model using GCNNs. Pose2Mesh reconstructs human mesh based on the given human pose representations.”
PARE: Part Attention Regressor for 3D Human Body Estimation (2021, methodology): “PARE is end-to-end trainable in a single stage, unlike recent multi-stage methods [10, 16, 37, 57].”
Human Pose Regression with Residual Log-likelihood Estimation (2021, methodology): “Two-stage methods [32, 13, 47, 68, 60, 9, 30, 67] lift the 2D poses to 3D space by regression.”
I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image (2020, background): “[7] proposed a graph convolutional network that recovers 3D human pose and mesh from a 2D human pose.”
Reconstructing Hands in 3D with Transformers (2023, background): “Performance on FreiHAND and HO3D tends to be more saturated and it is not surprising that the Method PA-MPJPE ↓ PA-MPVPE ↓ F@5 ↑ F@15 ↑ I2L-MeshNet [39] 7.4 7.6 0.681 0.973 Pose2Mesh [10] 7.7 7.8 0.674 0.969 I2UV-HandNet [7] 6.7 6.9 0.707 0.977 METRO [33] 6.5 6.3 0.731 0.984 Tang et al .”
CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation (2022, unknown): “Even with similar PA-MPJPE to Pose2Mesh [8], CLIFF reduces MPJPE by 12.”
Mesh Graphormer (2021, methodology): “Among the previous studies, the Graph Convolutional Neural Network (GCNN) [17, 5, 20] is one of the most popular options as it is able to model the local interactions between neighboring vertices based on a given adjacency matrix [32, 5, 20, 38].”
PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop (2021, methodology): “[10], silhouette ( i.e ., Fore-ground/Background, FB) and Part segmentation are considered in calculating the accuracy and f1 scores.”
MotionBERT: A Unified Perspective on Learning Human Motion Representations (2022, methodology): “Existing studies have shown the effectiveness of using 2D skeleton sequences for different downstream tasks [25, 32, 89, 109].”