Benchmarking Layout-Guided Diffusion Models through Unified Semantic-Spatial Evaluation in Closed and Open Settings

• URL: http://arxiv.org/abs/2604.25358v1
• Date: Tue, 28 Apr 2026 08:25:32 GMT
• Title: Benchmarking Layout-Guided Diffusion Models through Unified Semantic-Spatial Evaluation in Closed and Open Settings
• Abstract summary: We introduce a closed-set benchmark (C-Bench) to isolate key generative capabilities while providing varying levels of complexity in both prompt structure and layout.<n>To complement this controlled setting, we propose an open-set benchmark (O-Bench) that evaluates models using real-world prompts and layouts.<n>We conduct a large-scale evaluation of six state-of-the-art layout-guided diffusion models, totaling 319,086 generated and evaluated images.
• Score: 5.994477544297929
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Evaluating layout-guided text-to-image generative models requires assessing both semantic alignment with textual prompts and spatial fidelity to prescribed layouts. Assessing layout alignment requires collecting fine-grained annotations, which is costly and labor-intensive. Consequently, current benchmarks rarely provide comprehensive layout evaluation and often remain limited in scale or coverage, making model comparison, ranking, and interpretation difficult. In this work, we introduce a closed-set benchmark (C-Bench) designed to isolate key generative capabilities while providing varying levels of complexity in both prompt structure and layout. To complement this controlled setting, we propose an open-set benchmark (O-Bench) that evaluates models using real-world prompts and layouts, offering a measure of semantic and spatial alignment in the wild. We further develop a unified evaluation protocol that combines semantic and spatial accuracy into a single score, ensuring consistent model ranking. Using our benchmarks, we conduct a large-scale evaluation of six state-of-the-art layout-guided diffusion models, totaling 319,086 generated and evaluated images. We establish a model ranking based on their overall performance and provide detailed breakdowns for text and layout alignment to enhance interpretability. Fine-grained analyses across scenarios and prompt complexities highlight the strengths and limitations of current models. Code is available at https://github.com/lparolari/cobench.

Related papers

• Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach [58.15427952222424]
  General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples.<n>We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings.
  arXiv  Detail & Related papers  (2026-08-15T13:19:25Z)
• DualEval: Joint Model-Item Calibration for Unified LLM Evaluation [67.99119926012686]
  DualEval is a latent model-item calibration framework that represents models and evaluation items in a shared space.<n>We apply DualEval across four domains: coding, math, miscellaneous domain-knowledge tasks, and generic everyday user queries.
  arXiv  Detail & Related papers  (2026-06-24T22:40:46Z)
• DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark [48.84943754804533]
  Multimodal documents contain diverse elements, such as tables, figures, and layouts.<n>Current approaches typically combine dense visual embedding models with supervised rerankers to achieve high-precision retrieval.<n>We propose DocRetriever, a plug-and-play framework that enhances visual retrieval via a layout-aware sparse embedding technique.
  arXiv  Detail & Related papers  (2026-05-28T14:50:53Z)
• DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models [78.62380562116135]
  We propose DynT2I-Eval, a fully automated dynamic evaluation framework for text-to-image (T2I) models.<n>It constructs a structured visual semantic space from long-form descriptions, decomposing prompts into controllable dimensions.<n>DynT2I-Eval evaluates model performance across text alignment, perceptual quality, and aesthetics.
  arXiv  Detail & Related papers  (2026-05-07T12:53:51Z)
• SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation [62.55421542903781]
  We present textbfSpatialReward, a verifiable reward model explicitly designed to evaluate spatial layouts in generated images.<n>Experiments on Stable Diffusion and FLUX show that incorporating SpatialReward into RL training consistently improves spatial consistency and overall generation quality.
  arXiv  Detail & Related papers  (2026-03-23T17:26:35Z)
• IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation [85.56193980646981]
  We propose IF-RewardBench, a comprehensive meta-evaluation benchmark for instruction-following.<n>For each instruction, we construct a preference graph containing all pairwise preferences among multiple responses.<n>Experiments on IF-RewardBench reveal significant deficiencies in current judge models.
  arXiv  Detail & Related papers  (2026-03-05T02:21:17Z)
• StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models [98.72926158261937]
  We propose a training-free token pruning framework for Visual AutoRegressive models.<n>We employ a lightweight high-pass filter to capture local texture details, while leveraging Principal Component Analysis (PCA) to preserve global structural information.<n>To maintain valid next-scale prediction under sparse tokens, we introduce a nearest neighbor feature propagation strategy.
  arXiv  Detail & Related papers  (2026-03-02T11:35:05Z)
• Learning More from Less: Unlocking Internal Representations for Benchmark Compression [37.69575776639016]
  We introduce REPCORE, which aligns heterogeneous hidden states into a unified latent space to construct representative coresets.<n>Experiments on five benchmarks and over 200 models show consistent gains over output-based baselines in ranking correlation and estimation accuracy.
  arXiv  Detail & Related papers  (2026-01-31T13:11:39Z)
• REVEALER: Reinforcement-Guided Visual Reasoning for Element-Level Text-Image Alignment Evaluation [10.151027538362259]
  REVEALER is a unified framework for element-level alignment evaluation based on reinforcement-guided visual reasoning.<n>Our method enables Multimodal Large Language Models (MLLMs) to explicitly localize semantic elements and derive interpretable alignment judgments.
  arXiv  Detail & Related papers  (2025-12-29T03:24:09Z)
• CoT Referring: Improving Referring Expression Tasks with Grounded Reasoning [67.18702329644526]
  CoT Referring enhances model reasoning across modalities through a structured, chain-of-thought training data structure.<n>We restructure the training data to enforce a new output form, providing new annotations for existing datasets.<n>We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance.
  arXiv  Detail & Related papers  (2025-10-03T08:50:21Z)
• GRAFT: GRaPH and Table Reasoning for Textual Alignment -- A Benchmark for Structured Instruction Following and Visual Reasoning [0.0]
  GRAFT is a structured multimodal benchmark for evaluating models on instruction-following visual reasoning, and visual-textual alignment.<n>It features generated charts and synthetically rendered tables, created with Python libraries to ensure control over data semantics, and clarity.
  arXiv  Detail & Related papers  (2025-08-21T16:13:49Z)
• 7Bench: a Comprehensive Benchmark for Layout-guided Text-to-image Models [3.8123588214292745]
  We introduce 7Bench, the first benchmark to assess both semantic and spatial alignment in layout-guided text-to-image generation.<n>We propose an evaluation protocol that builds on existing frameworks by incorporating the layout alignment score to assess spatial accuracy.
  arXiv  Detail & Related papers  (2025-08-18T13:37:51Z)
• CountLoop: Training-Free High-Instance Image Generation via Iterative Agent Guidance [47.59187786346473]
  We present CountLoop, a training-free framework that provides diffusion models with accurate instance control.<n>Experiments on COCO Count, T2I CompBench, and two new high-instance benchmarks show that CountLoop achieves counting accuracy of up to 98%.
  arXiv  Detail & Related papers  (2025-08-18T11:28:02Z)
• Revisiting the Evaluation of Image Synthesis with GANs [55.72247435112475]
  This study presents an empirical investigation into the evaluation of synthesis performance, with generative adversarial networks (GANs) as a representative of generative models. In particular, we make in-depth analyses of various factors, including how to represent a data point in the representation space, how to calculate a fair distance using selected samples, and how many instances to use from each set.
  arXiv  Detail & Related papers  (2023-04-04T17:54:32Z)
• Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion [62.269219152425556]
  segmentation-based scene text detection methods have drawn extensive attention in the scene text detection field. We propose a Differentiable Binarization (DB) module that integrates the binarization process into a segmentation network. An efficient Adaptive Scale Fusion (ASF) module is proposed to improve the scale robustness by fusing features of different scales adaptively.
  arXiv  Detail & Related papers  (2022-02-21T15:30:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.