ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On

• URL: http://arxiv.org/abs/2606.27773v1
• Date: Fri, 26 Jun 2026 06:59:50 GMT
• Title: ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On
• Abstract summary: We present ModaFlow, a modality-aware flow-matching based framework for high-fidelity virtual try-on.<n>We show that ModaFlow achieves state-of-the-art results in both qualitative and quantitative evaluations.
• Score: 48.40260700745353
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image-based virtual try-on has emerged as a compelling task in e-commerce and augmented reality, yet existing methods struggle to simultaneously preserve fine garment semantics and adapt to diverse person body geometries under large clothing-body deformations. We present ModaFlow, a modality-aware flow-matching based framework for high-fidelity virtual try-on that achieves precise alignment between textual descriptions and garment appearance. Unlike prior methods that treat multimodal conditions uniformly, ModaFlow introduces a modality-aware guidance scheme: visual garment embeddings extracted by a pretrained image prompt adapter provide deterministic, persistent structural guidance, while textual embeddings generated from garment descriptions are controlled via classifier-free guidance (CFG) with adaptive scaling and zero-initialized velocity. To further enhance flow field accuracy, we propose two regularization losses, cosine similarity and perceptual flow discrimination, that jointly improve directional consistency and perceptual realism of the velocity field. Additionally, a mask manipulation strategy stochastically samples among box, transparent, and relaxed masks during training, simulating diverse occlusion scenarios and enabling robust inference under unpaired settings where only a box mask is available. Experiments show that ModaFlow achieves state-of-the-art results in both qualitative and quantitative evaluations, reducing FID by approximately 30% on paired and 20% on unpaired benchmarks.

Related papers

• FlowBender: Feedback-Aware Training for Self-Correcting Conditional Flows [18.746098727212544]
  FlowBender is a new diffusion and flow model for 3D reconstruction.<n>We show that FlowBender consistently outperforms standard supervised alignment training.<n>We propose several variants of FlowBender for differentiable operators.
  arXiv  Detail & Related papers  (2026-06-18T15:56:07Z)
• SubFlow: Sub-mode Conditioned Flow Matching for Diverse One-Step Generation [49.024920867668754]
  SubFlow is a flow matching framework that eliminates averaging distortion by each class into fine-grained sub-modes.<n>It integrates seamlessly into existing one-step models such as MeanFlow and Shortcut Models.<n>Experiments on ImageNet-256 demonstrate that SubFlow yields substantial gains in generation diversity (Recall) while maintaining competitive image quality (FID)
  arXiv  Detail & Related papers  (2026-04-14T04:36:02Z)
• JCo-MVTON: Jointly Controllable Multi-Modal Diffusion Transformer for Mask-Free Virtual Try-on [15.59886380067986]
  JCo-MVTON is a novel framework that overcomes limitations by integrating diffusion-based image generation with multi-modal conditional fusion.<n>It achieves state-of-the-art performance on public benchmarks including DressCode, significantly outperforming existing methods in both quantitative metrics and human evaluations.
  arXiv  Detail & Related papers  (2025-08-25T02:43:57Z)
• Voost: A Unified and Scalable Diffusion Transformer for Bidirectional Virtual Try-On and Try-Off [7.792838907882765]
  We propose Voost - a unified framework that jointly learns virtual try-on and try-off with a single diffusion transformer.<n>Voost achieves state-of-the-art results on both try-on and try-off benchmarks, consistently outperforming strong baselines in alignment accuracy, visual fidelity, and generalization.
  arXiv  Detail & Related papers  (2025-08-06T19:10:58Z)
• MF-VITON: High-Fidelity Mask-Free Virtual Try-On with Minimal Input [69.33864837012202]
  We propose a Mask-Free VITON framework that achieves realistic VITON using only a single person image and a target garment.<n>We leverage existing Mask-based VITON models to synthesize a high-quality dataset.<n>This dataset contains diverse, realistic pairs of person images and corresponding garments, augmented with varied backgrounds to mimic real-world scenarios.
  arXiv  Detail & Related papers  (2025-03-11T17:40:59Z)
• TryOn-Adapter: Efficient Fine-Grained Clothing Identity Adaptation for High-Fidelity Virtual Try-On [34.51850518458418]
  Virtual try-on focuses on adjusting the given clothes to fit a specific person seamlessly while avoiding any distortion of the patterns and textures of the garment. We propose an effective and efficient framework, termed TryOn-Adapter.
  arXiv  Detail & Related papers  (2024-04-01T03:15:41Z)
• Improving Diffusion Models for Authentic Virtual Try-on in the Wild [53.96244595495942]
  This paper considers image-based virtual try-on, which renders an image of a person wearing a curated garment. We propose a novel diffusion model that improves garment fidelity and generates authentic virtual try-on images. We present a customization method using a pair of person-garment images, which significantly improves fidelity and authenticity.
  arXiv  Detail & Related papers  (2024-03-08T08:12:18Z)
• DistractFlow: Improving Optical Flow Estimation via Realistic Distractions and Pseudo-Labeling [49.46842536813477]
  We propose a novel data augmentation approach, DistractFlow, for training optical flow estimation models. We combine one of the frames in the pair with a distractor image depicting a similar domain, which allows for inducing visual perturbations congruent with natural objects and scenes. Our approach allows increasing the number of available training pairs significantly without requiring additional annotations.
  arXiv  Detail & Related papers  (2023-03-24T15:42:54Z)
• Single Stage Virtual Try-on via Deformable Attention Flows [51.70606454288168]
  Virtual try-on aims to generate a photo-realistic fitting result given an in-shop garment and a reference person image. We develop a novel Deformable Attention Flow (DAFlow) which applies the deformable attention scheme to multi-flow estimation. Our proposed method achieves state-of-the-art performance both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2022-07-19T10:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.