PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop

By Hongwen Zhang, Yating Tian, Xinchi Zhou, Wanli Ouyang, Yebin Liu, Limin Wang, Zhenan Sun. IEEE International Conference on Computer Vision, 2021

393 citations recorded by the source corpus.

Regression-based methods have recently shown promising results in reconstructing human meshes from monocular images. By directly mapping raw pixels to model parameters, these methods can produce parametric models in a feed-forward manner via neural networks. However, minor deviation in parameters may lead to noticeable mis-alignment between the estimated meshes and image evidences. To address this issue, we propose a Pyramidal Mesh Alignment Feedback (PyMAF) loop to leverage a feature pyramid and rectify the predicted parameters explicitly based on the mesh-image alignment status in our deep regressor. In PyMAF, given the currently predicted parameters, mesh-aligned evidences will be extracted from finer-resolution features accordingly and fed back for parameter rectification. To reduce noise and enhance the reliability of these evidences, an auxiliary pixel-wise supervision is imposed on the feature encoder, which provides mesh-image correspondence guidance for our network to preserve the most related information in spatial features. The efficacy of our approach is validated on several benchmarks, including Human3.6M, 3DPW, LSP, and COCO, where experimental results show that our approach consistently improves the mesh-image alignment of the reconstruction. The project page with code and video results can be found at https://hongwenzhang.github.io/pymaf.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/e93f241b718c018b9ca514b1dd33236485fc0e05

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Learning-based Human Pose Estimation: A Survey (2020, background): “0 2021 [280] SMPL Image - - - - - - - - - - - - - - - 57.”
MotionBERT: A Unified Perspective on Learning Human Motion Representations (2022, methodology): “Based on the parametric human models such as SMPL [66], many research works [38,70,78, 115,125] focus on regressing the human mesh from a single image.”
ICON: Implicit Clothed humans Obtained from Normals (2021, methodology): “To estimate the SMPL mesh M ( β, θ ) ∈ R N × 3 from image I , we use PyMAF [68] due to its better mesh-to-image alignment compared to other methods.”
CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation (2022, methodology): “Optimization-based methods [4,11] are first proposed to iteratively fit the SMPL model to 2D evidences, while regression-based ones [18,62] make the predictions in a straightforward way that may support real time applications.”