PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction

By Zerong Zheng, Yebin Liu, Qionghai Dai. 2020

376 citations recorded by the source corpus.

Modeling 3D humans accurately and robustly from a single image is very challenging, and the key for such an ill-posed problem is the 3D representation of the human models. To overcome the limitations of regular 3D representations, we propose Parametric Model-Conditioned Implicit Representation (PaMIR), which combines the parametric body model with the free-form deep implicit function. In our PaMIR-based reconstruction framework, a novel deep neural network is proposed to regularize the free-form deep implicit function using the semantic features of the parametric model, which improves the generalization ability under the scenarios of challenging poses and various clothing topologies. Moreover, a novel depth-ambiguity-aware training loss is further integrated to resolve depth ambiguities and enable successful surface detail reconstruction with imperfect body reference. Finally, we propose a body reference optimization method to improve the parametric model estimation accuracy and to enhance the consistency between the parametric model and the implicit function. With the PaMIR representation, our framework can be easily extended to multi-image input scenarios without the need of multi-camera calibration and pose synchronization. Experimental results demonstrate that our method achieves state-of-the-art performance for image-based 3D human reconstruction in the cases of challenging poses and clothing types.

Subfields: Biomedical Engineering, Computational Mechanics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

1. Qionghai Dai
2. Yebin Liu
3. Dimitrios Tzionas
4. Michael J. Black
5. Yuliang Xiu
6. Zerong Zheng

In Computational Mechanics

1. Dimitrios Tzionas
2. Yebin Liu
3. Zerong Zheng
4. Qionghai Dai
5. Jinlong Yang
6. Yuliang Xiu
7. Michael J. Black
8. Kaiwen Guo

In Computer Vision and Pattern Recognition

1. Dimitrios Tzionas
2. Michael J. Black
3. Yebin Liu
4. Jinlong Yang
5. Qionghai Dai
6. Zerong Zheng
7. Yuliang Xiu
8. Pengpeng Liu

Citations in the corpus

• Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors (2021, methodology): “PaMIR [57] resolved this challenge by using the SMPL model as a prior but suffers from run-time inefficiency since it requires a post-processing optimization step.”
• ICON: Implicit Clothed humans Obtained from Normals (2021, methodology): “[54,55] 1000[70] 450[IC] 600[IC † ] 5[54,55] 150[IC] 375[24] 3109[IC † ] 600[70] 26[24]300[36,70] Table 1.”