Tex2Shape: Detailed Full Human Body Geometry From a Single Image

By Thiemo Alldieck, Gerard Pons-Moll, Christian Theobalt, Marcus A. Magnor. IEEE International Conference on Computer Vision, 2019

347 citations recorded by the source corpus.

We present a simple yet effective method to infer detailed full human body shape from only a single photograph. Our model can infer full-body shape including face, hair, and clothing including wrinkles at interactive frame-rates. Results feature details even on parts that are occluded in the input image. Our main idea is to turn shape regression into an aligned image-to-image translation problem. The input to our method is a partial texture map of the visible region obtained from off-the-shelf methods. From a partial texture, we estimate detailed normal and vector displacement maps, which can be applied to a low-resolution smooth body model to add detail and clothing. Despite being trained purely with synthetic data, our model generalizes well to real-world photographs. Numerous results demonstrate the versatility and robustness of our method.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/a2264b13076bba404f725dcb85c3f5290b03b42d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization (2020, methodology): “Then, ﬁne details represented as surface normal [51] or displacements [3] are added by either a post-process such as Shape From Shading [14] or composition within neural networks.”
PaMIR: Parametric Model-Conditioned Implicit Representation for Image-Based Human Reconstruction (2020, methodology): “We qualitatively compare our method with several state-of-the-art methods including HMD [3], Tex2Shape [4], Mould-ing Humans [6], DeepHuman [5] and PIFu [8].”
Multi-Garment Net: Learning to Dress 3D People From Images (2019, background): “[3, 5, 6] which reconstructs, from a single image or a video, clothing and hair as displacements on top of SMPL, but can not separate garments from body, and can not transfer clothing to new subjects.”
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion (2020, background): “More detail can be obtained predicting normals and displacement maps on a UV-map/geometry image of the surface [4, 37, 53].”
Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era (2019, methodology): “SMPL has the advantage of encoding in a disentangled manner both the shape, the pose, and the pose specific details, and thus it has been extensively used in deep learning-based human body shape reconstruction [110], [116], [117], [118].”
Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction (2020, background): “Predicting normals and displacement maps on a UV-map or geometry image of the surface [9,40] results in more detail, but predictions are not necessarily aligned with the observations.”
TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style (2020, background): “One way to achieve realism is to capture real clothing on humans from images [6, 5, 3, 4, 19], dynamic scans [41, 39] or RGBD [52, 51] and retarget it to novel shapes, but this is limited to re-animating the same motion on a novel shape.”
ARCH: Animatable Reconstruction of Clothed Humans (2020, background): “In recent work [19, 21, 39], the authors learn to generate surface geometry details and appearance using 2D UV maps.”
Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction (2020, methodology): “DeepHuman [29] and Tex2Shape [1] combine SMPL with methods of voxel reﬁnement and mesh deformation, respectively, for single-view clothed human mesh reconstruction.”
ICON: Implicit Clothed humans Obtained from Normals (2021, methodology): “There are now several methods that take a single image and regress a minimally clothed 3D human model [4,5,13,16,28,40].”
Neural Unsigned Distance Fields for Implicit Function Learning (2020, methodology): “Meshes are common to represent humans and have been used for estimating pose, shape [38, 40, 41, 52, 75, 85] and clothing [4, 3, 12] from images, but topologies and detail are restricted by an underlying model like SMPL [47] or SMPL + Garments meshes (ClothCap [57], TailorNet [54] and GarNet [30], S”
A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose (2021, positive): “This enables real-time reconstructions from single images [8, 18], detailed texturing and displacement mapping [4, 6], and alleviates manual rigging [3].”
Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensors (2021, neutral): “Methods in [58, 1] proposed to deform a parametric body model to fit the image observations including keypoints, silhouettes, and shading.”
Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies (2021, methodology): “To overcome this problem, [3, 2, 4] apply vertex displacements to the SMPL model to capture the human clothing and hair.”
Learning to Dress 3D People in Generative Clothing (2019, neutral): “Another group of methods are based on SMPL [1, 2, 3, 4, 8, 55].”