VITON: An Image-Based Virtual Try-on Network

By Xintong Han, Zuxuan Wu, Zhe Wu, Ruichi Yu, Larry S. Davis. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

763 citations recorded by the source corpus.

We present an image-based VIirtual Try-On Network (VITON) without using 3D information in any form, which seamlessly transfers a desired clothing item onto the corresponding region of a person using a coarse-to-fine strategy. Conditioned upon a new clothing-agnostic yet descriptive person representation, our framework first generates a coarse synthesized image with the target clothing item overlaid on that same person in the same pose. We further enhance the initial blurry clothing area with a refinement network. The network is trained to learn how much detail to utilize from the target clothing item, and where to apply to the person in order to synthesize a photo-realistic image in which the target item deforms naturally with clear visual patterns. Experiments on our newly collected Zalando dataset demonstrate its promise in the image-based virtual try-on task over state-of-the-art generative models.1

Subfields: Computational Mechanics, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/473993aad08d6e1967ad692513e7c539f07b780e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Photo-Realistic Virtual Try-On by Adaptively Generating↔Preserving Image Content (2020, methodology): “These two methods per-CA [19] VI [12] CP [42] VT [49] Comparison of representative virtual try-on methods.”
VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalization (2021, background): “In order to maintain the person’s shape, several approaches [10, 31, 36] provide a coarse body shape mask as a cue to synthesize the image, but fail to reproduce the body parts elab-orately ( e.g ., hands).”
Deep 3D human pose estimation: A review (2021, neutral): “Users can see how they look like when wearing a certain piece of clothing on the Internet in a virtual try-on system based on 3D pose estimation (Pons-Moll et al., 2017; Han et al., 2018).”
NISP: Pruning Networks Using Neuron Importance Score Propagation (2017, background): “Despite their impressive predictive power on a wide range of tasks [33, 40, 41, 13, 15, 48, 46, 24, 44, 42, 12, 45], the redundancy in the parameterization of deep learning models has been studied and demonstrated [6].”
MaskGAN: Towards Diverse and Interactive Facial Image Manipulation (2019, background): “Because GAN can generate realistic images, it enjoys pervasive applications on tasks such as image-to-image translation [14, 47, 25, 38, 30], image inpainting [24, 44, 45, 15], and virtual try-on [41, 9, 3, 37].”
Attribute Manipulation Generative Adversarial Networks for Fashion Images (2019, background): “The attention mask is combined with input and output images as in [12, 24, 33] where only specific regions are subject to attribute manipulations while the other regions are not changed.”
Toward Characteristic-Preserving Image-based Virtual Try-On Network (2018, methodology): “More recently, the image-based virtual try-on system [10] without resorting to 3D information, provides a more economical solution and shows promising results by reformulating it as a conditional image generation problem.”
Progressive Pose Attention Transfer for Person Image Generation (2019, background): “Several works [8, 39, 43, 31] were inspired by the virtually try-on applications and made good progress to change the clothes of a given person while maintaining the person pose by warping the clothes to fit for the body topology of the given person.”
VOGUE: Try-On by StyleGAN Interpolation Optimization (2021, methodology): “Given a pair of images (person, garment), the original VI-TON method [13] synthesizes a coarse try-on result, later reﬁned, and warped with thin plate splines over shape context matching [3].”
TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style (2020, methodology): “Recent methods use deep learning to generate pixels [20, 14, 13, 63, 65, 67, 21, 44, 54, 66], often guided by 2D warps, or smooth 3D models [30, 64, 59, 48], or learn to transfer texture from cloth images to 3D models [36].”