I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image

By Gyeongsik Moon, Kyoung Mu Lee. European Conference on Computer Vision, 2020

509 citations recorded by the source corpus.

Most of the previous image-based 3D human pose and mesh estimation methods estimate parameters of the human mesh model from an input image. However, directly regressing the parameters from the input image is a highly non-linear mapping because it breaks the spatial relationship between pixels in the input image. In addition, it cannot model the prediction uncertainty, which can make training harder. To resolve the above issues, we propose I2L-MeshNet, an image-to-lixel (line+pixel) prediction network. The proposed I2L-MeshNet predicts the per-lixel likelihood on 1D heatmaps for each mesh vertex coordinate instead of directly regressing the parameters. Our lixel-based 1D heatmap preserves the spatial relationship in the input image and models the prediction uncertainty. We demonstrate the benefit of the image-to-lixel prediction and show that the proposed I2L-MeshNet outperforms previous methods. The code is publicly available \footnote{\url{this https URL}}.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/ac22db90bfc80e67680a3b3bf6ca0f3af70fc9bb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

3D Human Pose Estimation with Spatial and Temporal Transformers (2021, methodology): “Direct estimation methods [31, 29] infer a 3D human pose from 2D images or video frames without intermediately estimating the 2D pose representation.”
Reconstructing Hands in 3D with Transformers (2023, background): “Performance on FreiHAND and HO3D tends to be more saturated and it is not surprising that the Method PA-MPJPE ↓ PA-MPVPE ↓ F@5 ↑ F@15 ↑ I2L-MeshNet [39] 7.4 7.6 0.681 0.973 Pose2Mesh [10] 7.7 7.8 0.674 0.969 I2UV-HandNet [7] 6.7 6.9 0.707 0.977 METRO [33] 6.5 6.3 0.731 0.984 Tang et al .”
HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation (2020, methodology): “The proposed approach is benchmarked in various 3D human pose and shape datasets, and it signiﬁcantly outperforms state-of-the-art approaches [24, 40] by 21.9 mm PVE on 3DPW [69], 6.6 mm PA-MPJPE on Human3.”
CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation (2022, unknown): “They could be modelbased [16,21] or model-free [34,26], and most of them are top-down methods, except for one bottom-up [49].”
PARE: Part Attention Regressor for 3D Human Body Estimation (2021, methodology): “PARE is end-to-end trainable in a single stage, unlike recent multi-stage methods [10, 16, 37, 57].”
MotionBERT: A Unified Perspective on Learning Human Motion Representations (2022, methodology): “Based on the parametric human models such as SMPL [66], many research works [38,70,78, 115,125] focus on regressing the human mesh from a single image.”
End-to-End Human Pose and Mesh Reconstruction with Transformers (2020, methodology): “As explored in literature [34, 22, 25, 24, 23, 8, 32], we use a mix-training strategy that leverages different training datasets, with or without the paired image-mesh annotations.”
Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose (2020, background): “Moon et al. [44] proposed a new heatmap representation, called lixel, to recover 3D human meshes.”
Mesh Graphormer (2021, methodology): “Therefore, we use the pseudo 3D mesh training data from [5, 26].”
Deep 3D human pose estimation: A review (2021, unknown): “Because of the small amount of information in a single image, most works adopt a parametric model like SMPL and learn to predict the shape and pose coefficients, such as (Arnab et al., 2019b; Sun et al., 2019; Sengupta et al., 2020; Choutas et al., 2020; Kolotouros et al., 2019b; Moon and Lee, 2020;”