Toward Characteristic-Preserving Image-based Virtual Try-On Network

By Bochao Wang, Huabing Zhang, Xiaodan Liang, Yimin Chen, Liang Lin, Meng Yang. European Conference on Computer Vision, 2018

492 citations recorded by the source corpus.

Image-based virtual try-on systems for fitting a new in-shop clothes into a person image have attracted increasing research attention, yet is still challenging. A desirable pipeline should not only transform the target clothes into the most fitting shape seamlessly but also preserve well the clothes identity in the generated image, that is, the key characteristics (e.g. texture, logo, embroidery) that depict the original clothes. However, previous image-conditioned generation works fail to meet these critical requirements towards the plausible virtual try-on performance since they fail to handle large spatial misalignment between the input image and target clothes. Prior work explicitly tackled spatial deformation using shape context matching, but failed to preserve clothing details due to its coarse-to-fine strategy. In this work, we propose a new fully-learnable Characteristic-Preserving Virtual Try-On Network (CP-VTON) for addressing all real-world challenges in this task. First, CP-VTON learns a thin-plate spline transformation for transforming the in-shop clothes into fitting the body shape of the target person via a new Geometric Matching Module (GMM) rather than computing correspondences of interest points as prior works did. Second, to alleviate boundary artifacts of warped clothes and make the results more realistic, we employ a Try-On Module that learns a composition mask to integrate the warped clothes and the rendered image to ensure smoothness. Extensive experiments on a fashion dataset demonstrate our CP-VTON achieves the state-of-the-art virtual try-on performance both qualitatively and quantitatively.

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/eff3ccb8a50e69d5839055dd1f3ddf21661ec60d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding (2023, unknown): “Customized image generation related to humans [15, 30, 37, 56] has received considerable attention, giving rise to numerous applications, such as personalized portrait photos [41], image animation [77], and virtual try-on [67].”
Progressive Pose Attention Transfer for Person Image Generation (2019, background): “Several works [8, 39, 43, 31] were inspired by the virtually try-on applications and made good progress to change the clothes of a given person while maintaining the person pose by warping the clothes to fit for the body topology of the given person.”
Attribute Manipulation Generative Adversarial Networks for Fashion Images (2019, unknown): “Generative networks have also been widely applied on various fashion-related tasks such as virtual try-on [15, 12, 30, 26] and fashion design/generation [19, 31, 29].”
Towards Photo-Realistic Virtual Try-On by Adaptively Generating↔Preserving Image Content (2020, methodology): “…matrix α (0 ≤ α ij ≤ 1) is then utilized to ﬁnally combine the two clothing images as the reﬁned clothing image T Rc by where (cid:12) denotes element-wise multiplication. α is also restricted by an regularization term (refer to CP-VTON [42]) and VGG loss is also introduced on T R c and T W c .”
VOGUE: Try-On by StyleGAN Interpolation Optimization (2021, methodology): “CP-VTON [30] adds geometric alignment to improve the details of the transferred garment.”
MaskGAN: Towards Diverse and Interactive Facial Image Manipulation (2019, background): “Because GAN can generate realistic images, it enjoys pervasive applications on tasks such as image-to-image translation [14, 47, 25, 38, 30], image inpainting [24, 44, 45, 15], and virtual try-on [41, 9, 3, 37].”
VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization (2021, methodology): “In order to maintain the person’s shape, several approaches [10, 31, 36] provide a coarse body shape mask as a cue to synthesize the image, but fail to reproduce the body parts elab-orately ( e.g ., hands).”
TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style (2020, methodology): “Recent methods use deep learning to generate pixels [20, 14, 13, 63, 65, 67, 21, 44, 54, 66], often guided by 2D warps, or smooth 3D models [30, 64, 59, 48], or learn to transfer texture from cloth images to 3D models [36].”