DeepHuman: 3D Human Reconstruction From a Single Image

By Zerong Zheng, Tao Yu, Yixuan Wei, Qionghai Dai, Yebin Liu. IEEE International Conference on Computer Vision, 2019

374 citations recorded by the source corpus.

We propose DeepHuman, an image-guided volume-to-volume translation CNN for 3D human reconstruction from a single RGB image. To reduce the ambiguities associated with the reconstruction of invisible areas, our method leverages a dense semantic representation generated from SMPL model as an additional input. One key feature of our network is that it fuses different scales of image features into the 3D space through volumetric feature transformation, which helps to recover accurate surface geometry. The surface details are further refined through a normal refinement network, which can be concatenated with the volume generation network using our proposed volumetric normal projection layer. We also contribute THuman, a 3D real-world human model dataset containing approximately 7000 models. The network is trained using training data generated from the dataset. Overall, due to the specific design of our network and the diversity in our dataset, our method enables 3D human model estimation given only a single image and outperforms state-of-the-art approaches.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/1a092cb522fa822960708584b181fd9cc4ed579c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies (2021, background): “Instead of optimizing the network parameters per scene, [42, 54, 67, 55] utilize networks to learn human shape priors from groundtruth 3D data, allowing them to reconstruct human shapes from even a single image.”
Tex2Shape: Detailed Full Human Body Geometry From a Single Image (2019, background): “In [61] the authors alleviate this limitation by augmenting the visible parts with a predicted normal map.”
Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans (2020, background): “Recently, [41, 47, 60, 48] capture the human prior from training data using networks, which enables them to recover 3D human geometry and texture from a single image.”
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization (2020, methodology): “Then, ﬁne details represented as surface normal [51] or displacements [3] are added by either a post-process such as Shape From Shading [14] or composition within neural networks.”
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction (2020, methodology): “Recently, DeepHuman [5] pioneered in conditioning the non-parametric volumetric representation on the parametric SMPL model, and demonstrates robust reconstruction for challenging poses and diverse garments.”
ARCH: Animatable Reconstruction of Clothed Humans (2020, background): “Recently, some progress [7, 15, 18, 20, 21, 23, 41, 51, 54] has been made in estimating parametric human bodies from a single RGB image, yet boundaries are under-explored to what extent 3D clothing details can be reconstructed from such inputs.”
ICON: Implicit Clothed humans Obtained from Normals (2021, methodology): “The “8x” setting is all 3 , 709 scans of AGORA [47] and THuman [71].”
Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction (2020, neutral): “CNN based reconstructions using voxels [51,18,62] or depthmaps [17,46,26] typically produce more details than parametric models, but limbs are often missing.”
PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop (2021, neutral): “In Table 3 of the main manuscript, the results are reported using mean AP, and variants of AP including AP 50 (AP at OKS = 0.50), AP 75 (AP at OKS = 0.75), AP M for persons with medium sizes, and AP L for persons with large sizes.”
Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors (2021, methodology): “DeepHuman [58] combined the parametric body model with a coarse-scale volumetric reconstruction network to reconstruct 3D human models from a single RGB image.”
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion (2020, methodology): “Humans with voxels: From images, CNN based reconstruction of humans represented as voxels [70, 17, 82] or depth-maps [16, 62, 38] typically produce more details than mesh or template-based representations, because predictions are aligned with the input pixels.”
Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction (2020, methodology): “DeepHuman [29] and Tex2Shape [1] combine SMPL with methods of voxel reﬁnement and mesh deformation, respectively, for single-view clothed human mesh reconstruction.”
Learning to Dress 3D People in Generative Clothing (2019, methodology): “† Group 1: BodyNet [48], DeepHuman [54], SiCloPe [34], PIFu [44], MouldingHumans [12].”