TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation

• URL: http://arxiv.org/abs/2608.16765v1
• Date: Mon, 17 Aug 2026 16:15:50 GMT
• Title: TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation
• Abstract summary: TRACE-Bench is an evaluation tool for multi-reference images spanning diverse artistic styles and real-world subjects.<n>It is built from 631 formula templates and around 4,000 reference images spanning diverse artistic styles and real-world subjects.<n>Its formula structure drives an operator-aligned evaluation protocol for per-capability scoring and a diagnostic tree analysis.
• Score: 74.69068352546073
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite recent advances in unified multimodal models for multi-reference image generation, existing benchmarks remain organized around predefined task types (e.g., "subject composition"), which are ill-suited to this combinatorial setting and lead to fragmented coverage, uncontrolled complexity, and little diagnostic value. Recognizing that diverse multi-reference tasks share a common set of atomic operations, we adopt a capability-oriented perspective and formalize four operators: Anchor ($f$), Disentangle ($g$), Apply ($\oplus$), and Compose ($C$). Any multi-reference prompt can then be represented as a compositional formula over these operators, whose structural complexity is quantified by the number of operator slots. Building on this formulation, we construct TRACE-Bench, comprising approximately 1,600 evaluation cases across slot counts 1--8, built from 631 formula templates and around 4,000 reference images spanning diverse artistic styles and real-world subjects. The formula structure directly drives an operator-aligned evaluation protocol for per-capability scoring and a diagnostic tree analysis for recursive failure localization. Evaluating 9 leading models reveals insights invisible to holistic scoring: the primary bottleneck lies in disentanglement ($g$) and attribute binding ($\oplus$) rather than scene-level composition ($C$), with even the best model scoring only 0.74 on attribute fidelity. Project page: https://amuseum-whr.github.io/TraceBench

Related papers

• Do Composed Image Retrieval Benchmarks Require Multimodal Composition? [86.99911649534795]
  We show that strong performance on Composed Image Retrieval benchmarks is assumed to require multimodal composition.<n>Across four widely used CIR benchmarks and eleven Generalist Multimodal Embedding models, a large fraction of queries can be solved using a single modality.<n>High CIR performance can arise from unimodal signals rather than true multimodal composition.
  arXiv  Detail & Related papers  (2026-05-14T12:56:36Z)
• DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification [38.65432183778467]
  We introduce DeltaRubric, an approach that reformulates multimodal preference evaluation as a plan-and-execute process within a single MLLM.<n>DeltaRubric operates in two steps: acting first as a $textitDisagreement Planner$, it generates a neutral, instance-specific verification checklist.<n>We formulate DeltaRubric as a multi-role reinforcement learning problem, jointly optimizing planning and verification capabilities.
  arXiv  Detail & Related papers  (2026-05-10T02:32:19Z)
• Deploy DINO with Many-to-Many Association [7.8674424739322175]
  We introduce a novel likelihood perspective, which interprets the existing method as a zeroth-order approximation of otherwise intractable likelihood calculation.<n>We propose a faster and finer-grained robust mechanism, termed as Harmonic Consensus Maximization (HCM)
  arXiv  Detail & Related papers  (2026-04-26T12:12:25Z)
• SCHEMA for Gemini 3 Pro Image: A Structured Methodology for Controlled AI Image Generation on Google's Native Multimodal Model [0.0]
  SCHEMA is an engineered framework built on systematic professional practice encompassing 850 verified API predictions.<n>SchEMA scales practitioner control from exploratory (approximately 5%) to directive (approximately 95%)<n>Key findings include an observed 91% Mandatory compliance rate and 94% Prohibitions compliance rate across 621 structured prompts.
  arXiv  Detail & Related papers  (2026-02-21T16:51:40Z)
• UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAG [82.84014669683863]
  Multimodal retrieval-augmented generation (MM-RAG) is a key approach for applying large language models to real-world knowledge bases.<n>UniDoc-Bench is the first large-scale, realistic benchmark for MM-RAG built from 70k real-world PDF pages.<n>Our experiments show that multimodal text-image fusion RAG systems consistently outperform both unimodal and jointly multimodal embedding-based retrieval.
  arXiv  Detail & Related papers  (2025-10-04T04:30:13Z)
• PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Models [80.65273820998875]
  We present PanMatch, a versatile foundation model for robust correspondence matching.<n>Our key insight is that any two-frame correspondence matching task can be addressed within a 2D displacement estimation framework.<n>PanMatch achieves multi-task integration by endowing displacement estimation algorithms with unprecedented generalization capabilities.
  arXiv  Detail & Related papers  (2025-07-11T08:18:52Z)
• With Limited Data for Multimodal Alignment, Let the STRUCTURE Guide You [3.3384874808567173]
  Multimodal models have demonstrated powerful capabilities in complex tasks requiring multimodal alignment, zero-shot classification and cross-modal retrieval.<n>Existing models typically rely on millions of paired samples, which are expensive or infeasible to obtain in domains.<n>We introduce an effective regularization technique that preserves the latent space of unimodal layers.<n>These two components can be readily incorporated into existing alignment methods, yielding substantial gains across 24 zero-shot image classification and retrieval benchmarks.
  arXiv  Detail & Related papers  (2025-06-20T10:32:54Z)
• T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation [55.16845189272573]
  T2I-CompBench++ is an enhanced benchmark for compositional text-to-image generation.<n>It comprises 8,000 compositional text prompts categorized into four primary groups: attribute binding, object relationships, generative numeracy, and complex compositions.
  arXiv  Detail & Related papers  (2023-07-12T17:59:42Z)
• SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
  arXiv  Detail & Related papers  (2021-04-19T11:21:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.