Learning to Dress 3D People in Generative Clothing

By Qianli Ma, Jinlong Yang, Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang, Michael J. Black. Computer Vision and Pattern Recognition, 2019

419 citations recorded by the source corpus.

Three-dimensional human body models are widely used in the analysis of human pose and motion. Existing models, however, are learned from minimally-clothed 3D scans and thus do not generalize to the complexity of dressed people in common images and videos. Additionally, current models lack the expressive power needed to represent the complex non-linear geometry of pose-dependent clothing shapes. To address this, we learn a generative 3D mesh model of clothed people from 3D scans with varying pose and clothing. Specifically, we train a conditional Mesh-VAE-GAN to learn the clothing deformation from the SMPL body model, making clothing an additional term in SMPL. Our model is conditioned on both pose and clothing type, giving the ability to draw samples of clothing to dress different body shapes in a variety of styles and poses. To preserve wrinkle detail, our Mesh-VAE-GAN extends patchwise discriminators to 3D meshes. Our model, named CAPE, represents global shape and fine local structure, effectively extending the SMPL body model to clothing. To our knowledge, this is the first generative model that directly dresses 3D human body meshes and generalizes to different poses. The model, code and data are available for research purposes at https://cape.is.tue.mpg.de.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/d5381bfb45abf0f92a7344deb9d530963fa91408

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

In Control and Systems Engineering

Citations in the corpus

Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance (2024, background): “Its widespread adoption spans fields like human reconstruction [1, 13] and interaction with environments [12, 23].”
NeuMan: Neural Human Radiance Field from a Single Video (2022, methodology): “We hope to apply more expressive body models to model hand gestures [31], even garment dynamics [24], to mitigate this problem.”
TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style (2020, methodology): “While learning pose-dependent models from real captures [61, 29, 33] is definitely an exciting route, accurately capturing sufﬁcient real data is still a major challenge.”
ICON: Implicit Clothed humans Obtained from Normals (2021, methodology): “We evaluate primarily on CAPE [35], which no method uses for training, to test their generelizability.”