Towards Photo-Realistic Virtual Try-On by Adaptively Generating↔Preserving Image Content

By Han Yang, Ruimao Zhang, Xiaobao Guo, Wei David Liu, Wangmeng Zuo, Ping Luo. Computer Vision and Pattern Recognition, 2020

337 citations recorded by the source corpus.

Image visual try-on aims at transferring a target clothes image onto a reference person, and has become a hot topic in recent years. Prior arts usually focus on preserving the character of a clothes image (e.g. texture, logo, embroidery) when warping it to arbitrary human pose. However, it remains a big challenge to generate photo-realistic try-on images when large occlusions and human poses are presented in the reference person. To address this issue, we propose a novel visual try-on network, namely Adaptive Content Generating and Preserving Network (ACGPN). In particular, ACGPN first predicts semantic layout of the reference image that will be changed after try-on (e.g.long sleeve shirt→arm, arm→jacket), and then determines whether its image content needs to be generated or preserved according to the predicted semantic layout, leading to photo-realistic try-on and rich clothes details. ACGPN generally involves three major modules. First, a semantic layout generation module utilizes semantic segmentation of the reference image to progressively predict the desired semantic layout after try-on. Second, a clothes warping module warps clothes image according to the generated semantic layout, where a second-order difference constraint is introduced to stabilize the warping process during training.Third, an inpainting module for content fusion integrates all information (e.g. reference image, semantic layout, warped clothes) to adaptively produce each semantic part of human body. In comparison to the state-of-the-art methods, ACGPN can generate photo-realistic images with much better perceptual quality and richer fine-details.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/803cf4834ada9662bcf9317f8a312cf87ea8a424

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

MaskGAN: Towards Diverse and Interactive Facial Image Manipulation (2019, background): “Because GAN can generate realistic images, it enjoys pervasive applications on tasks such as image-to-image translation [14, 47, 25, 38, 30], image inpainting [24, 44, 45, 15], and virtual try-on [41, 9, 3, 37].”
VOGUE: Try-On by StyleGAN Interpolation Optimization (2021, background): “ACGPN [32] aims to preserve the target person’s identity in addition to the transferred clothes details, by accounting for semantic layout.”
VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization (2021, methodology): “Also, several recent methods [9, 36, 35] add a module that generates segmentation maps and determine the person’s layout from the ﬁnal image in advance.”