GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning

• URL: http://arxiv.org/abs/2303.13756v1
• Date: Fri, 24 Mar 2023 02:12:29 GMT
• Title: GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning
• Authors: Zhenyu Xie and Zaiyu Huang and Xin Dong and Fuwei Zhao and Haoye Dong and Xijin Zhang and Feida Zhu and Xiaodan Liang
• Abstract summary: Virtual Try-ON aims to transfer an in-shop garment onto a specific person. Existing methods employ a global warping module to model the anisotropic deformation for different garment parts. We propose an innovative Local-Flow Global-Parsing (LFGP) warping module and a Dynamic Gradient Truncation (DGT) training strategy.
• Score: 63.8668179362151
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image-based Virtual Try-ON aims to transfer an in-shop garment onto a specific person. Existing methods employ a global warping module to model the anisotropic deformation for different garment parts, which fails to preserve the semantic information of different parts when receiving challenging inputs (e.g, intricate human poses, difficult garments). Moreover, most of them directly warp the input garment to align with the boundary of the preserved region, which usually requires texture squeezing to meet the boundary shape constraint and thus leads to texture distortion. The above inferior performance hinders existing methods from real-world applications. To address these problems and take a step towards real-world virtual try-on, we propose a General-Purpose Virtual Try-ON framework, named GP-VTON, by developing an innovative Local-Flow Global-Parsing (LFGP) warping module and a Dynamic Gradient Truncation (DGT) training strategy. Specifically, compared with the previous global warping mechanism, LFGP employs local flows to warp garments parts individually, and assembles the local warped results via the global garment parsing, resulting in reasonable warped parts and a semantic-correct intact garment even with challenging inputs.On the other hand, our DGT training strategy dynamically truncates the gradient in the overlap area and the warped garment is no more required to meet the boundary constraint, which effectively avoids the texture squeezing problem. Furthermore, our GP-VTON can be easily extended to multi-category scenario and jointly trained by using data from different garment categories. Extensive experiments on two high-resolution benchmarks demonstrate our superiority over the existing state-of-the-art methods.

Related papers

• StyDeSty: Min-Max Stylization and Destylization for Single Domain Generalization [85.18995948334592]
  Single domain generalization (single DG) aims at learning a robust model generalizable to unseen domains from only one training domain. State-of-the-art approaches have mostly relied on data augmentations, such as adversarial perturbation and style enhancement, to synthesize new data. We propose emphStyDeSty, which explicitly accounts for the alignment of the source and pseudo domains in the process of data augmentation.
  arXiv  Detail & Related papers  (2024-06-01T02:41:34Z)
• GC-VTON: Predicting Globally Consistent and Occlusion Aware Local Flows with Neighborhood Integrity Preservation for Virtual Try-on [0.0]
  Flow based garment warping is an integral part of image-based virtual try-on networks. We propose a novel approach where we disentangle the global boundary alignment and local texture preserving tasks. A consistency loss is then employed between the two modules which harmonizes the local flows with the global boundary alignment.
  arXiv  Detail & Related papers  (2023-11-07T10:09:49Z)
• GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization [52.55628139825667]
  Federated Learning (FL) has emerged as a promising distributed machine learning framework to preserve clients' privacy. Recent studies find that an attacker can invert the shared gradients and recover sensitive data against an FL system by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD), which disassembles the GAN model and searches the feature domains of the intermediate layers.
  arXiv  Detail & Related papers  (2023-08-09T04:34:21Z)
• PG-VTON: A Novel Image-Based Virtual Try-On Method via Progressive Inference Paradigm [6.929743379017671]
  We propose a novel virtual try-on method via progressive inference paradigm (PGVTON) We exploit the try-on parsing as the shape guidance and implement the garment try-on via warping-mapping-composition. Experiments demonstrate that our method has state-of-the-art performance under two challenging scenarios.
  arXiv  Detail & Related papers  (2023-04-18T12:47:26Z)
• PASTA-GAN++: A Versatile Framework for High-Resolution Unpaired Virtual Try-on [70.12285433529998]
  PASTA-GAN++ is a versatile system for high-resolution unpaired virtual try-on. It supports unsupervised training, arbitrary garment categories, and controllable garment editing.
  arXiv  Detail & Related papers  (2022-07-27T11:47:49Z)
• Style-Based Global Appearance Flow for Virtual Try-On [119.95115739956661]
  A novel global appearance flow estimation model is proposed in this work. Experiment results on a popular virtual try-on benchmark show that our method achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-04-03T10:58:04Z)
• Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN [66.3650689395967]
  We propose a texture-preserving end-to-end network, the PAtch-routed SpaTially-Adaptive GAN (PASTA-GAN), that facilitates real-world unpaired virtual try-on. To disentangle the style and spatial information of each garment, PASTA-GAN consists of an innovative patch-routed disentanglement module.
  arXiv  Detail & Related papers  (2021-11-20T08:36:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.