Multi-Garment Net: Learning to Dress 3D People From Images

By Bharat Lal Bhatnagar, Garvita Tiwari, Christian Theobalt, Gerard Pons-Moll. IEEE International Conference on Computer Vision, 2019

465 citations recorded by the source corpus.

We present Multi-Garment Network (MGN), a method to predict body shape and clothing, layered on top of the SMPL model from a few frames (1-8) of a video. Several experiments demonstrate that this representation allows higher level of control when compared to single mesh or voxel representations of shape. Our model allows to predict garment geometry, relate it to the body shape, and transfer it to new body shapes and poses. To train MGN, we leverage a digital wardrobe containing 712 digital garments in correspondence, obtained with a novel method to register a set of clothing templates to a dataset of real 3D scans of people in different clothing and poses. Garments from the digital wardrobe, or predicted by MGN, can be used to dress any body shape in arbitrary poses. We will make publicly available the digital wardrobe, the MGN model, and code to dress SMPL with the garments at https://virtualhumans.mpi-inf.mpg.de/mgn

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/aa08a7b0d2fa0877bc13793f113a7c8dbc31da66

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

BEHAVE: Dataset and Method for Tracking Human Object Interactions (2022, background): “The last decade has seen rapid progress in modelling the appearance of humans ranging from body pose, shape [52, 58, 60, 61, 81], faces [74] and even detailed clothing [5, 7, 11,57,65].”
Neural Unsigned Distance Fields for Implicit Function Learning (2020, methodology): “7) and open surfaces (307 garments from MGN [11], see Fig.”
PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization (2020, methodology): “While using a separate parameteric model can alleviate the limited shape variation [6], large deformations and topological changes are still non-trivial to handle with these shape representations.”
ICON: Implicit Clothed humans Obtained from Normals (2021, methodology): “To address this, some methods [8, 22] use a classifier to identify cloth types in the input image, and then perform cloth-aware inference for 3D reconstruction.”
Image-Based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era (2019, unknown): “Some methods in both categories reconstruct naked 3D human body shapes [108], [109], while others recover also cloths and garments [110], [111].”
Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction (2020, methodology): “Hybrid methods mitigate these problems by refining feed-forward predictions with optimization at training[57] and/or test time[59], and by predicting displacements on top of SMPL, demonstrating capture of fine details and even clothing [6,11].”
Learning to Dress 3D People in Generative Clothing (2019, methodology): “In contrast to similar models that also dress SMPLwith offsets [1, 8], our garment layer is parametrized, low-dimensional, and pose-dependent.”
ARCH: Animatable Reconstruction of Clothed Humans (2020, background): “There are multiple attempts to solve this task with video inputs [2, 3, 37, 1, 52], RGB-D data [53, 56] and in multi-view settings [5, 13, 14, 45, 46, 47, 48, 6].”
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion (2020, methodology): “Since SMPL does not model clothing and detail, recent methods predict deformations from SMPL [1, 2, 3, 51, 7] or a template [21, 22].”