T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation

• URL: http://arxiv.org/abs/2508.17472v1
• Date: Sun, 24 Aug 2025 17:59:38 GMT
• Title: T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation
• Abstract summary: We propose T2I-ReasonBench, a benchmark evaluating reasoning capabilities of text-to-image (T2I) models.<n>It consists of four dimensions: Textual Image Design, Entity-Reasoning and Scientific-Reasoning.
• Score: 41.3139483775936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose T2I-ReasonBench, a benchmark evaluating reasoning capabilities of text-to-image (T2I) models. It consists of four dimensions: Idiom Interpretation, Textual Image Design, Entity-Reasoning and Scientific-Reasoning. We propose a two-stage evaluation protocol to assess the reasoning accuracy and image quality. We benchmark various T2I generation models, and provide comprehensive analysis on their performances.

Related papers

• Faithful, Enriched, and Precise: Benchmarking Natural-Science Illustration Generation by T2I models [49.93401412148884]
  We introduce FEPBench, a benchmark built from carefully selected high-quality scientific illustrations.<n>We evaluate T2I models along three dimensions: instruction faithfulness, reasoning enrichment, and semantic precision.<n>Results show that even state-of-the-art closed-source models, such as GPT Image 2 and Nano Banana Pro, still suffer from text-rendering bottlenecks, limited reasoning enrichment, and difficulty balancing generation richness with precision.
  arXiv  Detail & Related papers  (2026-06-04T09:49:02Z)
• AcT2I: Evaluating and Improving Action Depiction in Text-to-Image Models [58.85362281293525]
  We introduce AcT2I, a benchmark designed to evaluate the performance of T2I models in generating images from action-centric prompts.<n>We experimentally validate that leading T2I models do not fare well on AcT2I.<n>We build upon this by developing a training-free, knowledge distillation technique utilizing Large Language Models to address this limitation.
  arXiv  Detail & Related papers  (2025-09-19T16:41:39Z)
• OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation [23.05106664412349]
  Text-to-image (T2I) models have garnered significant attention for generating high-quality images aligned with text prompts.<n>OneIG-Bench is a benchmark framework for evaluation of T2I models across multiple dimensions.
  arXiv  Detail & Related papers  (2025-06-09T17:50:21Z)
• TIIF-Bench: How Does Your T2I Model Follow Your Instructions? [7.13169573900556]
  We present TIIF-Bench (Text-to-Image Instruction Following Benchmark), aiming to systematically assess T2I models' ability in interpreting and following intricate textual instructions.<n> TIIF-Bench comprises a set of 5000 prompts organized along multiple dimensions, which are categorized into three levels of difficulties and complexities.<n>Two critical attributes, i.e. text rendering and style control, are introduced to evaluate the precision of text synthesis and the aesthetic coherence of T2I models.
  arXiv  Detail & Related papers  (2025-06-02T18:44:07Z)
• R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation [26.816674696050413]
  Reasoning is a fundamental capability often required in real-world text-to-image (T2I) generation.<n>Recent T2I models have made impressive progress in producing photorealistic images, but their reasoning capability remains underdeveloped.<n>We introduce R2I-Bench, a benchmark specifically designed to rigorously assess reasoning-driven T2I generation.
  arXiv  Detail & Related papers  (2025-05-29T14:43:46Z)
• IA-T2I: Internet-Augmented Text-to-Image Generation [13.765327654914199]
  Current text-to-image (T2I) generation models achieve promising results, but they fail on the scenarios where the knowledge implied in the text prompt is uncertain.<n>We propose an Internet-Augmented text-to-image generation (IA-T2I) framework to compel T2I models clear about such uncertain knowledge by providing them with reference images.
  arXiv  Detail & Related papers  (2025-05-21T17:31:49Z)
• PhyBench: A Physical Commonsense Benchmark for Evaluating Text-to-Image Models [50.33699462106502]
  Text-to-image (T2I) models frequently fail to produce images consistent with physical commonsense. Current T2I evaluation benchmarks focus on metrics such as accuracy, bias, and safety, neglecting the evaluation of models' internal knowledge. We introduce PhyBench, a comprehensive T2I evaluation dataset comprising 700 prompts across 4 primary categories: mechanics, optics, thermodynamics, and material properties.
  arXiv  Detail & Related papers  (2024-06-17T17:49:01Z)
• Lost in Translation? Translation Errors and Challenges for Fair Assessment of Text-to-Image Models on Multilingual Concepts [107.32683485639654]
  Benchmarks of the multilingual capabilities of text-to-image (T2I) models compare generated images prompted in a test language to an expected image distribution over a concept set. One such benchmark, "Conceptual Coverage Across Languages" (CoCo-CroLa), assesses the tangible noun inventory of T2I models by prompting them to generate pictures from a concept list translated to seven languages. We find that this benchmark contains translation errors of varying severity in Spanish, Japanese, and Chinese.
  arXiv  Detail & Related papers  (2024-03-17T05:05:11Z)
• A Contrastive Compositional Benchmark for Text-to-Image Synthesis: A Study with Unified Text-to-Image Fidelity Metrics [58.83242220266935]
  We introduce Winoground-T2I, a benchmark designed to evaluate the compositionality of T2I models. This benchmark includes 11K complex, high-quality contrastive sentence pairs spanning 20 categories. We use Winoground-T2I with a dual objective: to evaluate the performance of T2I models and the metrics used for their evaluation.
  arXiv  Detail & Related papers  (2023-12-04T20:47:48Z)
• DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual Design [124.56730013968543]
  We introduce DEsignBench, a text-to-image (T2I) generation benchmark tailored for visual design scenarios. For DEsignBench benchmarking, we perform human evaluations on generated images against the criteria of image-text alignment, visual aesthetic, and design creativity. In addition to human evaluations, we introduce the first automatic image generation evaluator powered by GPT-4V.
  arXiv  Detail & Related papers  (2023-10-23T17:48:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.