TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style

By Chaitanya Patel, Zhouyingcheng Liao, Gerard Pons-Moll. Computer Vision and Pattern Recognition, 2020

315 citations recorded by the source corpus.

In this paper, we present TailorNet, a neural model which predicts clothing deformation in 3D as a function of three factors: pose, shape and style (garment geometry), while retaining wrinkle detail. This goes beyond prior models, which are either specific to one style and shape, or generalize to different shapes producing smooth results, despite being style specific. Our hypothesis is that (even non-linear) combinations of examples smoothes out high frequency components such as fine-wrinkles, which makes learning the three factors jointly hard. At the heart of our technique is a decomposition of deformation into a high frequency and a low frequency component. While the low-frequency component is predicted from pose, shape and style parameters with an MLP, the high-frequency component is predicted with a mixture of shape-style specific pose models. The weights of the mixture are computed with a narrow bandwidth kernel to guarantee that only predictions with similar high-frequency patterns are combined. The style variation is obtained by computing, in a canonical pose, a subspace of deformation, which satisfies physical constraints such as inter-penetration, and draping on the body. TailorNet delivers 3D garments which retain the wrinkles from the physics based simulations (PBS) it is learned from, while running more than 1000 times faster. In contrast to classical PBS, TailorNet is easy to use and fully differentiable, which is crucial for computer vision and learning algorithms. Several experiments demonstrate TailorNet produces more realistic results than prior work, and even generates temporally coherent deformations on sequences of the AMASS dataset, despite being trained on static poses from a different dataset. To stimulate further research in this direction, we will make a dataset consisting of 55800 frames, as well as our model publicly available at https://virtualhumans.mpi-inf.mpg.de/tailornet/.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/c6b052b5b288d2a1ef4877c5ca63cd1481ed2b7b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Neural Unsigned Distance Fields for Implicit Function Learning (2020, methodology): “…to represent humans and have been used for estimating pose, shape [38, 40, 41, 52, 76, 86] and clothing [3, 2, 11] from images, but topologies and detail are restricted by an underlying model like SMPL [47] or SMPL + Garments meshes (ClothCap [57], TailorNet [54] and GarNet [30], SIZER [74]).”
VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization (2021, background): “There are two main categories for virtual try-on approaches: 3D model-based approaches [8, 27, 23, 22] and 2D image-based approaches [10, 31, 9, 36, 35, 5].”
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion (2020, background): “However, all these approaches require different templates [7, 49] for every new garment topology or do not produce high-quality reconstructions [53].”