VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-On

• URL: http://arxiv.org/abs/2603.11734v1
• Date: Thu, 12 Mar 2026 09:38:46 GMT
• Title: VTEdit-Bench: A Comprehensive Benchmark for Multi-Reference Image Editing Models in Virtual Try-On
• Abstract summary: Universal multi-reference image editing models have progressed rapidly and exhibit strong generalization in visual editing.<n>We introduce VTEdit-Bench, a benchmark designed to evaluate universal multi-reference image editing models across various realistic VTON scenarios.<n>We also propose VTEdit-QA, a reference-aware VLM-based evaluator that assesses VTON performance from three key aspects: model consistency, cloth consistency, and overall image quality.
• Score: 39.307333593250995
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As virtual try-on (VTON) continues to advance, a growing number of real-world scenarios have emerged, pushing beyond the ability of the existing specialized VTON models. Meanwhile, universal multi-reference image editing models have progressed rapidly and exhibit strong generalization in visual editing, suggesting a promising route toward more flexible VTON systems. However, despite their strong capabilities, the strengths and limitations of universal editors for VTON remain insufficiently explored due to the lack of systematic evaluation benchmarks. To address this gap, we introduce VTEdit-Bench, a comprehensive benchmark designed to evaluate universal multi-reference image editing models across various realistic VTON scenarios. VTEdit-Bench contains 24,220 test image pairs spanning five representative VTON tasks with progressively increasing complexity, enabling systematic analysis of robustness and generalization. We further propose VTEdit-QA, a reference-aware VLM-based evaluator that assesses VTON performance from three key aspects: model consistency, cloth consistency, and overall image quality. Through this framework, we systematically evaluate eight universal editing models and compare them with seven specialized VTON models. Results show that top universal editors are competitive on conventional tasks and generalize more stably to harder scenarios, but remain challenged by complex reference configurations, particularly multi-cloth conditioning.

Related papers

• PROMO: Promptable Outfitting for Efficient High-Fidelity Virtual Try-On [39.47263059671598]
  VTON is a structured image editing problem that demands strong conditional generation under three key requirements.<n>We present PROMO, a promptable virtual try-on framework built upon a Flow Matching DiT backbone with latent multi-modal conditional concatenation.<n>On standard benchmarks, PROMO surpasses both prior VTON methods and general image editing models in visual fidelity while delivering a competitive balance between quality and speed.
  arXiv  Detail & Related papers  (2026-03-12T08:41:05Z)
• How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing [56.60465182650588]
  We introduce three-level interaction hierarchy that captures deictic grounding, morphological manipulation, and causal reasoning.<n>We propose a robust LMM-as-a-judge evaluation framework with task-specific metrics to enable scalable and fine-grained assessment.<n>We find that proprietary models exhibit early-stage visual instruction-following capabilities and consistently outperform open-source models.
  arXiv  Detail & Related papers  (2026-02-02T09:24:45Z)
• VTONGuard: Automatic Detection and Authentication of AI-Generated Virtual Try-On Content [18.869017622630647]
  We present VTONGuard, a benchmark dataset containing over 775,000 real and synthetic try-on images.<n>The dataset covers diverse real-world conditions, including variations in pose, background, and garment styles.<n>Our results reveal each method's strengths and weaknesses and highlight the persistent challenge of cross-paradigm generalization.
  arXiv  Detail & Related papers  (2026-01-20T13:26:06Z)
• VTONQA: A Multi-Dimensional Quality Assessment Dataset for Virtual Try-on [83.39966045949338]
  VTONQA is the first multi-dimensional quality assessment dataset specifically designed for VTON.<n>It contains 8,132 images generated by 11 representative VTON models, along with 24,396 mean opinion scores (MOSs) across three evaluation dimensions.<n>We benchmark both VTON models and a diverse set of image quality assessment (IQA) metrics, revealing the limitations of existing methods.
  arXiv  Detail & Related papers  (2026-01-06T11:42:26Z)
• UniREditBench: A Unified Reasoning-based Image Editing Benchmark [52.54256348710893]
  This work proposes UniREditBench, a unified benchmark for reasoning-based image editing evaluation.<n>It comprises 2,700 meticulously curated samples, covering both real- and game-world scenarios across 8 primary dimensions and 18 sub-dimensions.<n>We fine-tune Bagel on this dataset and develop UniREdit-Bagel, demonstrating substantial improvements in both in-domain and out-of-distribution settings.
  arXiv  Detail & Related papers  (2025-11-03T07:24:57Z)
• EyeSim-VQA: A Free-Energy-Guided Eye Simulation Framework for Video Quality Assessment [68.77813885751308]
  EyeSimVQA is a novel VQA framework that incorporates free-energy-based self-repair.<n>We show EyeSimVQA achieves competitive or superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-06-13T08:00:54Z)
• VTBench: Evaluating Visual Tokenizers for Autoregressive Image Generation [11.529598741483076]
  A visual tokenizer (VT) maps continuous pixel inputs to discrete token sequences.<n>Current discrete VTs fall significantly behind continuous variational autoencoders (VAEs), leading to degraded image reconstructions and poor preservation of details and text.<n>Existing benchmarks focus on end-to-end generation quality, without isolating VT performance.<n>We introduce VTBench, a comprehensive benchmark that systematically evaluates VTs across three core tasks: Image Reconstruction, Detail Preservation, and Text Preservation.
  arXiv  Detail & Related papers  (2025-05-19T17:59:01Z)
• Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution [82.38677987249348]
  We present the Qwen2-VL Series, which redefines the conventional predetermined-resolution approach in visual processing. Qwen2-VL introduces the Naive Dynamic Resolution mechanism, which enables the model to dynamically process images of varying resolutions into different numbers of visual tokens. The model also integrates Multimodal Rotary Position Embedding (M-RoPE), facilitating the effective fusion of positional information across text, images, and videos.
  arXiv  Detail & Related papers  (2024-09-18T17:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.