From Keypoints to Realism: A Realistic and Accurate Virtual Try-on Network from 2D Images

• URL: http://arxiv.org/abs/2504.03807v1
• Date: Fri, 04 Apr 2025 10:35:06 GMT
• Title: From Keypoints to Realism: A Realistic and Accurate Virtual Try-on Network from 2D Images
• Authors: Maliheh Toozandehjani, Ali Mousavi, Reza Taheri,
• Abstract summary: The aim of image-based virtual try-on is to generate realistic images of individuals wearing target garments.<n>The generator produces the final image with high visual quality, reconstructing the precise characteristics of the target garment.
• Score: 4.2578780373293235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The aim of image-based virtual try-on is to generate realistic images of individuals wearing target garments, ensuring that the pose, body shape and characteristics of the target garment are accurately preserved. Existing methods often fail to reproduce the fine details of target garments effectively and lack generalizability to new scenarios. In the proposed method, the person's initial garment is completely removed. Subsequently, a precise warping is performed using the predicted keypoints to fully align the target garment with the body structure and pose of the individual. Based on the warped garment, a body segmentation map is more accurately predicted. Then, using an alignment-aware segment normalization, the misaligned areas between the warped garment and the predicted garment region in the segmentation map are removed. Finally, the generator produces the final image with high visual quality, reconstructing the precise characteristics of the target garment, including its overall shape and texture. This approach emphasizes preserving garment characteristics and improving adaptability to various poses, providing better generalization for diverse applications.

Related papers

• Limb-Aware Virtual Try-On Network with Progressive Clothing Warping [64.84181064722084]
  Image-based virtual try-on aims to transfer an in-shop clothing image to a person image.<n>Most existing methods adopt a single global deformation to perform clothing warping directly.<n>We propose Limb-aware Virtual Try-on Network named PL-VTON, which performs fine-grained clothing warping progressively.
  arXiv  Detail & Related papers  (2025-03-18T09:52:41Z)
• FitDiT: Advancing the Authentic Garment Details for High-fidelity Virtual Try-on [73.13242624924814]
  Garment perception enhancement technique, FitDiT, is designed for high-fidelity virtual try-on using Diffusion Transformers (DiT) We introduce a garment texture extractor that incorporates garment priors evolution to fine-tune garment feature, facilitating to better capture rich details such as stripes, patterns, and text. We also employ a dilated-relaxed mask strategy that adapts to the correct length of garments, preventing the generation of garments that fill the entire mask area during cross-category try-on.
  arXiv  Detail & Related papers  (2024-11-15T11:02:23Z)
• Better Fit: Accommodate Variations in Clothing Types for Virtual Try-on [25.550019373321653]
  Image-based virtual try-on aims to transfer target in-shop clothing to a dressed model image. We propose an adaptive mask training paradigm that dynamically adjusts training masks. For unpaired try-on validation, we construct a comprehensive cross-try-on benchmark.
  arXiv  Detail & Related papers  (2024-03-13T12:07:14Z)
• StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On [35.227896906556026]
  Given a clothing image and a person image, an image-based virtual try-on aims to generate a customized image that appears natural and accurately reflects the characteristics of the clothing image. In this work, we aim to expand the applicability of the pre-trained diffusion model so that it can be utilized independently for the virtual try-on task. Our proposed zero cross-attention blocks not only preserve the clothing details by learning the semantic correspondence but also generate high-fidelity images by utilizing the inherent knowledge of the pre-trained model in the warping process.
  arXiv  Detail & Related papers  (2023-12-04T08:27:59Z)
• Style-Based Global Appearance Flow for Virtual Try-On [119.95115739956661]
  A novel global appearance flow estimation model is proposed in this work. Experiment results on a popular virtual try-on benchmark show that our method achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-04-03T10:58:04Z)
• Arbitrary Virtual Try-On Network: Characteristics Preservation and Trade-off between Body and Clothing [85.74977256940855]
  We propose an Arbitrary Virtual Try-On Network (AVTON) for all-type clothes. AVTON can synthesize realistic try-on images by preserving and trading off characteristics of the target clothes and the reference person. Our approach can achieve better performance compared with the state-of-the-art virtual try-on methods.
  arXiv  Detail & Related papers  (2021-11-24T08:59:56Z)
• Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN [66.3650689395967]
  We propose a texture-preserving end-to-end network, the PAtch-routed SpaTially-Adaptive GAN (PASTA-GAN), that facilitates real-world unpaired virtual try-on. To disentangle the style and spatial information of each garment, PASTA-GAN consists of an innovative patch-routed disentanglement module.
  arXiv  Detail & Related papers  (2021-11-20T08:36:12Z)
• SPG-VTON: Semantic Prediction Guidance for Multi-pose Virtual Try-on [27.870740623131816]
  Image-based virtual try-on is challenging in fitting a target in-shop clothes into a reference person under diverse human poses. We propose an end-to-end Semantic Prediction Guidance multi-pose Virtual Try-On Network (SPG-VTON) We evaluate the proposed method on the most massive multi-pose dataset (MPV) and the DeepFashion dataset.
  arXiv  Detail & Related papers  (2021-08-03T15:40:50Z)
• VOGUE: Try-On by StyleGAN Interpolation Optimization [14.327659393182204]
  Given an image of a target person and an image of another person wearing a garment, we automatically generate the target garment. At the core of our method is a pose-conditioned StyleGAN2 latent space, which seamlessly combines the areas of interest from each image. Our algorithm allows for garments to deform according to the given body shape, while preserving pattern and material details.
  arXiv  Detail & Related papers  (2021-01-06T22:01:46Z)
• Towards Photo-Realistic Virtual Try-On by Adaptively Generating$\leftrightarrow$Preserving Image Content [85.24260811659094]
  We propose a novel visual try-on network, namely Adaptive Content Generating and Preserving Network (ACGPN) ACGPN first predicts semantic layout of the reference image that will be changed after try-on. Second, a clothes warping module warps clothing images according to the generated semantic layout. Third, an inpainting module for content fusion integrates all information (e.g. reference image, semantic layout, warped clothes) to adaptively produce each semantic part of human body.
  arXiv  Detail & Related papers  (2020-03-12T15:55:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.