Learning Garment DensePose for Robust Warping in Virtual Try-On
- URL: http://arxiv.org/abs/2303.17688v1
- Date: Thu, 30 Mar 2023 20:02:29 GMT
- Title: Learning Garment DensePose for Robust Warping in Virtual Try-On
- Authors: Aiyu Cui, Sen He, Tao Xiang, Antoine Toisoul
- Abstract summary: We propose a robust warping method for virtual try-on based on a learned garment DensePose.
Our method achieves the state-of-the-art equivalent on virtual try-on benchmarks.
- Score: 72.13052519560462
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Virtual try-on, i.e making people virtually try new garments, is an active
research area in computer vision with great commercial applications. Current
virtual try-on methods usually work in a two-stage pipeline. First, the garment
image is warped on the person's pose using a flow estimation network. Then in
the second stage, the warped garment is fused with the person image to render a
new try-on image. Unfortunately, such methods are heavily dependent on the
quality of the garment warping which often fails when dealing with hard poses
(e.g., a person lifting or crossing arms). In this work, we propose a robust
warping method for virtual try-on based on a learned garment DensePose which
has a direct correspondence with the person's DensePose. Due to the lack of
annotated data, we show how to leverage an off-the-shelf person DensePose model
and a pretrained flow model to learn the garment DensePose in a weakly
supervised manner. The garment DensePose allows a robust warping to any
person's pose without any additional computation. Our method achieves the
state-of-the-art equivalent on virtual try-on benchmarks and shows warping
robustness on in-the-wild person images with hard poses, making it more suited
for real-world virtual try-on applications.
Related papers
- PocoLoco: A Point Cloud Diffusion Model of Human Shape in Loose Clothing [97.83361232792214]
PocoLoco is the first template-free, point-based, pose-conditioned generative model for 3D humans in loose clothing.
We formulate avatar clothing deformation as a conditional point-cloud generation task within the denoising diffusion framework.
We release a dataset of two subjects performing various poses in loose clothing with a total of 75K point clouds.
arXiv Detail & Related papers (2024-11-06T20:42:13Z) - Better Fit: Accommodate Variations in Clothing Types for Virtual Try-on [25.550019373321653]
Image-based virtual try-on aims to transfer target in-shop clothing to a dressed model image.
We propose an adaptive mask training paradigm that dynamically adjusts training masks.
For unpaired try-on validation, we construct a comprehensive cross-try-on benchmark.
arXiv Detail & Related papers (2024-03-13T12:07:14Z) - Improving Diffusion Models for Authentic Virtual Try-on in the Wild [53.96244595495942]
This paper considers image-based virtual try-on, which renders an image of a person wearing a curated garment.
We propose a novel diffusion model that improves garment fidelity and generates authentic virtual try-on images.
We present a customization method using a pair of person-garment images, which significantly improves fidelity and authenticity.
arXiv Detail & Related papers (2024-03-08T08:12:18Z) - AniDress: Animatable Loose-Dressed Avatar from Sparse Views Using
Garment Rigging Model [58.035758145894846]
We introduce AniDress, a novel method for generating animatable human avatars in loose clothes using very sparse multi-view videos.
A pose-driven deformable neural radiance field conditioned on both body and garment motions is introduced, providing explicit control of both parts.
Our method is able to render natural garment dynamics that deviate highly from the body and well to generalize to both unseen views and poses.
arXiv Detail & Related papers (2024-01-27T08:48:18Z) - Street TryOn: Learning In-the-Wild Virtual Try-On from Unpaired Person Images [14.616371216662227]
We introduce a StreetTryOn benchmark to support in-the-wild virtual try-on applications.
We also propose a novel method to learn virtual try-on from a set of in-the-wild person images directly without requiring paired data.
arXiv Detail & Related papers (2023-11-27T18:59:02Z) - Fill in Fabrics: Body-Aware Self-Supervised Inpainting for Image-Based
Virtual Try-On [3.5698678013121334]
We propose a self-supervised conditional generative adversarial network based framework comprised of a Fabricator and a Segmenter, Warper and Fuser.
The Fabricator reconstructs the clothing image when provided with a masked clothing as input, and learns the overall structure of the clothing by filling in fabrics.
A virtual try-on pipeline is then trained by transferring the learned representations from the Fabricator to Warper in an effort to warp and refine the target clothing.
arXiv Detail & Related papers (2022-10-03T13:25:31Z) - Significance of Skeleton-based Features in Virtual Try-On [3.7552180803118325]
The idea of textitVirtual Try-ON (VTON) benefits e-retailing by giving an user the convenience of trying a clothing at the comfort of their home.
Most of the existing VTON methods produce inconsistent results when a person posing with his arms folded.
We propose two learning-based modules: a synthesizer network and a mask prediction network.
arXiv Detail & Related papers (2022-08-17T05:24:03Z) - Style-Based Global Appearance Flow for Virtual Try-On [119.95115739956661]
A novel global appearance flow estimation model is proposed in this work.
Experiment results on a popular virtual try-on benchmark show that our method achieves new state-of-the-art performance.
arXiv Detail & Related papers (2022-04-03T10:58:04Z) - Dressing in the Wild by Watching Dance Videos [69.7692630502019]
This paper attends to virtual try-on in real-world scenes and brings improvements in authenticity and naturalness.
We propose a novel generative network called wFlow that can effectively push up garment transfer to in-the-wild context.
arXiv Detail & Related papers (2022-03-29T08:05:45Z) - LGVTON: A Landmark Guided Approach to Virtual Try-On [4.617329011921226]
Given the images of two people: a person and a model, it generates a rendition of the person wearing the clothes of the model.
This is useful considering the fact that on most e-commerce websites images of only clothes are not usually available.
arXiv Detail & Related papers (2020-04-01T16:49:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.