Rethinking Garment Conditioning in Diffusion-based Virtual Try-On

• URL: http://arxiv.org/abs/2511.18775v1
• Date: Mon, 24 Nov 2025 05:19:44 GMT
• Title: Rethinking Garment Conditioning in Diffusion-based Virtual Try-On
• Authors: Kihyun Na, Jinyoung Choi, Injung Kim,
• Abstract summary: We develop Re-CatVTON, an efficient single UNet model that achieves high performance.<n>The proposed Re-CatVTON significantly improves performance compared to its predecessor.<n>Our results demonstrate improved FID, KID, and LPIPS scores, with only a marginal decrease in SSIM.
• Score: 7.386027762996787
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Virtual Try-On (VTON) is the task of synthesizing an image of a person wearing a target garment, conditioned on a person image and a garment image. While diffusion-based VTON models featuring a Dual UNet architecture demonstrate superior fidelity compared to single UNet models, they incur substantial computational and memory overhead due to their heavy structure. In this study, through visualization analysis and theoretical analysis, we derived three hypotheses regarding the learning of context features to condition the denoising process. Based on these hypotheses, we developed Re-CatVTON, an efficient single UNet model that achieves high performance. We further enhance the model by introducing a modified classifier-free guidance strategy tailored for VTON's spatial concatenation conditioning, and by directly injecting the ground-truth garment latent derived from the clean garment latent to prevent the accumulation of prediction error. The proposed Re-CatVTON significantly improves performance compared to its predecessor (CatVTON) and requires less computation and memory than the high-performance Dual UNet model, Leffa. Our results demonstrate improved FID, KID, and LPIPS scores, with only a marginal decrease in SSIM, establishing a new efficiency-performance trade-off for single UNet VTON models.

Related papers

• SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution [46.311223206965934]
  We study key design principles for latter cascaded video super-resolution models, which are underexplored currently.<n>First, we propose two strategies to generate training pairs that better mimic the output characteristics of the base model, ensuring alignment between the VSR model and its upstream generator.<n>Second, we provide critical insights into VSR model behavior through systematic analysis of (1) timestep sampling strategies, (2) noise augmentation effects on low-resolution (LR) inputs.
  arXiv  Detail & Related papers  (2025-06-24T17:57:26Z)
• DS-VTON: An Enhanced Dual-Scale Coarse-to-Fine Framework for Virtual Try-On [33.05238077456732]
  We propose DS-VTON, an enhanced dual-scale coarse-to-fine framework for virtual try-on.<n> DS-VTON consists of two stages: the first generates a low-resolution try-on result to capture the semantic correspondence between garment and body.<n>In the second stage, a blend-refine diffusion process reconstructs high-resolution outputs by refining the residual between scales.
  arXiv  Detail & Related papers  (2025-06-01T08:52:57Z)
• One-Step Diffusion Model for Image Motion-Deblurring [85.76149042561507]
  We propose a one-step diffusion model for deblurring (OSDD), a novel framework that reduces the denoising process to a single step.<n>To tackle fidelity loss in diffusion models, we introduce an enhanced variational autoencoder (eVAE), which improves structural restoration.<n>Our method achieves strong performance on both full and no-reference metrics.
  arXiv  Detail & Related papers  (2025-03-09T09:39:57Z)
• Simplifying CLIP: Unleashing the Power of Large-Scale Models on Consumer-level Computers [3.2492319522383717]
  Contrastive Language-Image Pre-training (CLIP) has attracted a surge of attention for its superior zero-shot performance and excellent transferability to downstream tasks.<n>However, training such large-scale models usually requires substantial computation and storage, which poses barriers for general users with consumer-level computers.
  arXiv  Detail & Related papers  (2024-11-22T08:17:46Z)
• Stable Consistency Tuning: Understanding and Improving Consistency Models [40.2712218203989]
  Diffusion models achieve superior generation quality but suffer from slow generation speed due to iterative nature of denoising.<n> consistency models, a new generative family, achieve competitive performance with significantly faster sampling.<n>We propose a novel framework for understanding consistency models by modeling the denoising process of the diffusion model as a Markov Decision Process (MDP) and framing consistency model training as the value estimation through Temporal Difference(TD) Learning.
  arXiv  Detail & Related papers  (2024-10-24T17:55:52Z)
• CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models [77.39903417768967]
  CatVTON is a virtual try-on diffusion model that transfers in-shop or worn garments of arbitrary categories to target individuals.<n>CatVTON consists only of a VAE and a simplified denoising UNet, removing redundant image and text encoders.<n>Experiments demonstrate that CatVTON achieves superior qualitative and quantitative results compared to baseline methods.
  arXiv  Detail & Related papers  (2024-07-21T11:58:53Z)
• Distilling Diffusion Models into Conditional GANs [90.76040478677609]
  We distill a complex multistep diffusion model into a single-step conditional GAN student model. For efficient regression loss, we propose E-LatentLPIPS, a perceptual loss operating directly in diffusion model's latent space. We demonstrate that our one-step generator outperforms cutting-edge one-step diffusion distillation models.
  arXiv  Detail & Related papers  (2024-05-09T17:59:40Z)
• Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation [59.184980778643464]
  Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI) In this paper, we introduce an innovative technique called self-play fine-tuning for diffusion models (SPIN-Diffusion) Our approach offers an alternative to conventional supervised fine-tuning and RL strategies, significantly improving both model performance and alignment.
  arXiv  Detail & Related papers  (2024-02-15T18:59:18Z)
• A-SDM: Accelerating Stable Diffusion through Redundancy Removal and Performance Optimization [54.113083217869516]
  In this work, we first explore the computational redundancy part of the network. We then prune the redundancy blocks of the model and maintain the network performance. Thirdly, we propose a global-regional interactive (GRI) attention to speed up the computationally intensive attention part.
  arXiv  Detail & Related papers  (2023-12-24T15:37:47Z)
• JoReS-Diff: Joint Retinex and Semantic Priors in Diffusion Model for Low-light Image Enhancement [69.6035373784027]
  Low-light image enhancement (LLIE) has achieved promising performance by employing conditional diffusion models. Previous methods may neglect the importance of a sufficient formulation of task-specific condition strategy. We propose JoReS-Diff, a novel approach that incorporates Retinex- and semantic-based priors as the additional pre-processing condition.
  arXiv  Detail & Related papers  (2023-12-20T08:05:57Z)
• Variational Diffusion Models [33.0719137062396]
  We introduce a family of diffusion-based generative models that obtain state-of-the-art likelihoods on image density estimation benchmarks. We show that the variational lower bound (VLB) simplifies to a remarkably short expression in terms of the signal-to-noise ratio of the diffused data.
  arXiv  Detail & Related papers  (2021-07-01T17:43:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.