Real-Time Per-Garment Virtual Try-On with Temporal Consistency for Loose-Fitting Garments

• URL: http://arxiv.org/abs/2506.12348v1
• Date: Sat, 14 Jun 2025 04:57:21 GMT
• Title: Real-Time Per-Garment Virtual Try-On with Temporal Consistency for Loose-Fitting Garments
• Abstract summary: We propose a two-stage approach for robust semantic map estimation.<n>First, we extract a garment-invariant representation from the raw input image.<n>This representation is then passed through an auxiliary network to estimate the semantic map.
• Score: 11.732243977359412
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Per-garment virtual try-on methods collect garment-specific datasets and train networks tailored to each garment to achieve superior results. However, these approaches often struggle with loose-fitting garments due to two key limitations: (1) They rely on human body semantic maps to align garments with the body, but these maps become unreliable when body contours are obscured by loose-fitting garments, resulting in degraded outcomes; (2) They train garment synthesis networks on a per-frame basis without utilizing temporal information, leading to noticeable jittering artifacts. To address these challenges, we propose a two-stage approach for robust semantic map estimation. First, we extract a garment-invariant representation from the raw input image. This representation is then passed through an auxiliary network to estimate the semantic map. This enhances the robustness of semantic map estimation under loose-fitting garments during garment-specific dataset generation. Furthermore, we introduce a recurrent garment synthesis framework that incorporates temporal dependencies to improve frame-to-frame coherence while maintaining real-time performance. We conducted qualitative and quantitative evaluations to demonstrate that our method outperforms existing approaches in both image quality and temporal coherence. Ablation studies further validate the effectiveness of the garment-invariant representation and the recurrent synthesis framework.

Related papers

• DualFit: A Two-Stage Virtual Try-On via Warping and Synthesis [8.082593574401704]
  We propose DualFit to preserve fine-grained garment details such as logos and printed text elements.<n>In the first stage, DualFit warps the target garment to align with the person image using a learned flow field.<n>In the second stage, a fidelity-fidelity try-on module synthesizes the final output by blending the warped garment with preserved human regions.
  arXiv  Detail & Related papers  (2025-08-16T18:50:31Z)
• DiffFit: Disentangled Garment Warping and Texture Refinement for Virtual Try-On [3.5655800569257896]
  Virtual try-on (VTON) aims to synthesize realistic images of a person wearing a target garment, with broad applications in e-commerce and digital fashion.<n>We propose DiffFit, a novel two-stage latent diffusion framework for high-fidelity virtual try-on.
  arXiv  Detail & Related papers  (2025-06-29T15:31:42Z)
• HiGarment: Cross-modal Harmony Based Diffusion Model for Flat Sketch to Realistic Garment Image [20.177936034245572]
  HiGarment is a novel framework that enhances fabric representation across textual and visual modalities.<n>We propose Flat Sketch to Realistic Garment Image (FS2RG), which generates realistic garment images by integrating flat sketches and textual guidance.<n>We collect Multi-modal Detailed Garment, the largest open-source dataset for garment generation.
  arXiv  Detail & Related papers  (2025-05-29T07:23:40Z)
• MagicTryOn: Harnessing Diffusion Transformer for Garment-Preserving Video Virtual Try-on [28.66545985357718]
  Virtual Try-On (VVT) aims to synthesize garments that appear natural across consecutive frames, capturing both their dynamics and interactions with human cues.<n>Existing VVT methods still suffer from inadequate garment fidelity and limitedtemporal consistency.<n>We present MagicTryOn, a diffusion-transformer based framework for garment-constrained video virtual try-on.
  arXiv  Detail & Related papers  (2025-05-27T15:22:02Z)
• Beyond Static Perception: Integrating Temporal Context into VLMs for Cloth Folding [6.0914269533287175]
  BiFold is a model that predicts language-conditioned pick-and-place actions from visual observations.<n>We present evidence that its fine-tuning and temporal context enable effective alignment between text and image regions.
  arXiv  Detail & Related papers  (2025-05-12T14:24:03Z)
• GarmentX: Autoregressive Parametric Representations for High-Fidelity 3D Garment Generation [15.345904761472106]
  GarmentX is a novel framework for generating diverse, high-fidelity, and wearable 3D garments from a single input image.<n>We introduce GarmentX dataset, a large-scale dataset of 378,682 garment parameter-image pairs.
  arXiv  Detail & Related papers  (2025-04-29T04:15:33Z)
• Towards Robust and Realistic Human Pose Estimation via WiFi Signals [85.60557095666934]
  WiFi-based human pose estimation is a challenging task that bridges discrete and subtle WiFi signals to human skeletons.<n>This paper revisits this problem and reveals two critical yet overlooked issues: 1) cross-domain gap, i.e., due to significant variations between source-target domain pose distributions; and 2) structural fidelity gap, i.e., predicted skeletal poses manifest distorted topology.<n>This paper fills these gaps by reformulating the task into a novel two-phase framework dubbed DT-Pose: Domain-consistent representation learning and Topology-constrained Pose decoding.
  arXiv  Detail & Related papers  (2025-01-16T09:38:22Z)
• IGR: Improving Diffusion Model for Garment Restoration from Person Image [6.384713545839356]
  Garment restoration, the inverse of virtual try-on task, focuses on restoring standard garment from a person image.<n>We propose an improved diffusion model for restoring authentic garments.<n>Our approach employs two garment extractors to independently capture low-level features and high-level semantics from the person image.
  arXiv  Detail & Related papers  (2024-12-16T07:48:30Z)
• FitDiT: Advancing the Authentic Garment Details for High-fidelity Virtual Try-on [73.13242624924814]
  Garment perception enhancement technique, FitDiT, is designed for high-fidelity virtual try-on using Diffusion Transformers (DiT) We introduce a garment texture extractor that incorporates garment priors evolution to fine-tune garment feature, facilitating to better capture rich details such as stripes, patterns, and text. We also employ a dilated-relaxed mask strategy that adapts to the correct length of garments, preventing the generation of garments that fill the entire mask area during cross-category try-on.
  arXiv  Detail & Related papers  (2024-11-15T11:02:23Z)
• Towards Garment Sewing Pattern Reconstruction from a Single Image [76.97825595711444]
  Garment sewing pattern represents the intrinsic rest shape of a garment, and is the core for many applications like fashion design, virtual try-on, and digital avatars. We first synthesize a versatile dataset, named SewFactory, which consists of around 1M images and ground-truth sewing patterns. We then propose a two-level Transformer network called Sewformer, which significantly improves the sewing pattern prediction performance.
  arXiv  Detail & Related papers  (2023-11-07T18:59:51Z)
• HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamics [84.29846699151288]
  Our method is agnostic to body shape and applies to tight-fitting garments as well as loose, free-flowing clothing. As one key contribution, we propose a hierarchical message-passing scheme that efficiently propagates stiff stretching modes.
  arXiv  Detail & Related papers  (2022-12-14T14:24:00Z)
• Direct Embedding of Temporal Network Edges via Time-Decayed Line Graphs [51.51417735550026]
  Methods for machine learning on temporal networks generally exhibit at least one of two limitations. We present a simple method that avoids both shortcomings: construct the line graph of the network, which includes a node for each interaction, and weigh the edges of this graph based on the difference in time between interactions. Empirical results on real-world networks demonstrate our method's efficacy and efficiency on both edge classification and temporal link prediction.
  arXiv  Detail & Related papers  (2022-09-30T18:24:13Z)
• DIG: Draping Implicit Garment over the Human Body [56.68349332089129]
  We propose an end-to-end differentiable pipeline that represents garments using implicit surfaces and learns a skinning field conditioned on shape and pose parameters of an articulated body model. We show that our method, thanks to its end-to-end differentiability, allows to recover body and garments parameters jointly from image observations.
  arXiv  Detail & Related papers  (2022-09-22T08:13:59Z)
• Leveraging Self-Supervision for Cross-Domain Crowd Counting [71.75102529797549]
  State-of-the-art methods for counting people in crowded scenes rely on deep networks to estimate crowd density. We train our network to recognize upside-down real images from regular ones and incorporate into it the ability to predict its own uncertainty. This yields an algorithm that consistently outperforms state-of-the-art cross-domain crowd counting ones without any extra computation at inference time.
  arXiv  Detail & Related papers  (2021-03-30T12:37:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.