TangPoetryBench: A Multi-Dimensional Benchmark and Rubric-Conditioned Evaluator for Poetry-to-Image Generation

• URL: http://arxiv.org/abs/2608.11452v1
• Date: Tue, 11 Aug 2026 21:30:53 GMT
• Title: TangPoetryBench: A Multi-Dimensional Benchmark and Rubric-Conditioned Evaluator for Poetry-to-Image Generation
• Abstract summary: TangBench is a benchmark of 1,280 images (320 classical Chinese Tang poems x 4 state-of-the-art T2I models) with quality-controlled human annotations across ten dimensions.<n>We reveal the shared and model-specific strengths and weaknesses of current T2I models, including their ability to evoke a poem's implicit emotion.<n>We also introduce PoemAutoEvaluator (PAE), an open, rubric-conditioned evaluator that reaches parity with a strong proprietary judge.
• Score: 3.970365166505577
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-image (T2I) models are increasingly asked to illustrate literary and cultural content, yet we cannot measure how well an image renders the meaning of a poem. The task is many-sided: a good illustration must be visually sound, faithful to the poem's imagery and scene, culturally and stylistically apt, free of spurious text, and true to its emotion, and its deepest requirements, imagery and especially implicit emotion, are never stated in the words. Existing metrics (CLIPScore, BLIPScore, VQAScore) reward literal text-image correspondence and so cannot tell whether an illustration succeeds, let alone why, or even separate the best model from the worst. We introduce TangPoetryBench, a multi-dimensional benchmark of 1,280 images (320 classical Chinese Tang poems x 4 state-of-the-art T2I models) with quality-controlled human annotations across ten dimensions. Analyzing this data, we reveal the shared and model-specific strengths and weaknesses of current T2I models, including their ability to evoke a poem's implicit emotion. We further introduce PoemAutoEvaluator (PAE), an open, rubric-conditioned evaluator that reaches parity with a strong proprietary judge (Claude), generalizes to an unseen generator and a second poetic tradition (Song Ci), and lets the benchmark scale to new images without fresh human annotation. We release the benchmark, annotations, and evaluator.

Related papers

• Do It Yourself (DIY): Modifying Images for Poems in a Zero-Shot Setting Using Weighted Prompt Manipulation [20.357558748582942]
  We introduce a novel Weighted Prompt Manipulation (WPM) technique, which modifies attention weights and text embeddings within diffusion models.<n>WPM enhances or suppresses their influence in the final generated image, leading to semantically richer and more contextually accurate visualizations.<n>This is the first attempt at integrating weighted prompt manipulation for enhancing imagery in poetic language.
  arXiv  Detail & Related papers  (2025-09-15T12:58:38Z)
• PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinement [18.293592213622183]
  PoemTale Diffusion aims to minimise the information that is lost during poetic text-to-image conversion.<n>To support this, we adapt existing state-of-the-art diffusion models by modifying their self-attention mechanisms.<n>To encourage research in the field of poetry, we introduce the P4I dataset, consisting of 1111 poems.
  arXiv  Detail & Related papers  (2025-07-18T07:33:08Z)
• RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning [88.14234949860105]
  RePrompt is a novel reprompting framework that introduces explicit reasoning into the prompt enhancement process via reinforcement learning.<n>Our approach enables end-to-end training without human-annotated data.
  arXiv  Detail & Related papers  (2025-05-23T06:44:26Z)
• Replace in Translation: Boost Concept Alignment in Counterfactual Text-to-Image [53.09546752700792]
  We propose a strategy to instruct this replacing process, which is called as Explicit Logical Narrative Prompt (ELNP)<n>We design a metric to calculate how many required concepts in the prompt can be covered averagely in the synthesized images.<n>The extensive experiments and qualitative comparisons demonstrate that our strategy can boost the concept alignment in counterfactual T2I.
  arXiv  Detail & Related papers  (2025-05-20T13:27:52Z)
• Poetry in Pixels: Prompt Tuning for Poem Image Generation via Diffusion Models [18.293592213622183]
  We propose a PoemToPixel framework designed to generate images that visually represent the inherent meanings of poems.<n>Our approach incorporates the concept of prompt tuning in our image generation framework to ensure that the resulting images closely align with the poetic content.<n>To expand the diversity of the poetry dataset, we introduce MiniPo, a novel multimodal dataset comprising 1001 children's poems and images.
  arXiv  Detail & Related papers  (2025-01-10T10:26:54Z)
• Sonnet or Not, Bot? Poetry Evaluation for Large Models and Datasets [3.0040661953201475]
  Large language models (LLMs) can now generate and recognize poetry. We develop a task to evaluate how well LLMs recognize one aspect of English-language poetry. We show that state-of-the-art LLMs can successfully identify both common and uncommon fixed poetic forms.
  arXiv  Detail & Related papers  (2024-06-27T05:36:53Z)
• Poetry2Image: An Iterative Correction Framework for Images Generated from Chinese Classical Poetry [7.536700229966157]
  Poetry2Image is an iterative correction framework for images generated from Chinese classical poetry. The proposed method achieves an average element completeness of 70.63%, representing an improvement of 25.56% over direct image generation.
  arXiv  Detail & Related papers  (2024-06-15T19:45:08Z)
• Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense? [97.0899853256201]
  We present a novel task and benchmark for evaluating the ability of text-to-image generation models to produce images that align with commonsense in real life. We evaluate whether T2I models can conduct visual-commonsense reasoning, e.g. produce images that fit "the lightbulb is unlit" vs. "the lightbulb is lit" We benchmark a variety of state-of-the-art (sota) T2I models and surprisingly find that, there is still a large gap between image synthesis and real life photos.
  arXiv  Detail & Related papers  (2024-06-11T17:59:48Z)
• Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models [63.545146807810305]
  Text-to-image diffusion models can generate high-quality pictures from textual input prompts. These models have been trained using text data collected from content-based labelling protocols. We characterise the sentimentality, objectiveness and degree of abstraction of publicly available text data used to train current text-to-image diffusion models.
  arXiv  Detail & Related papers  (2022-10-19T14:20:05Z)
• Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest [70.40189243067857]
  Large neural networks can now generate jokes, but do they really "understand" humor? We challenge AI models with three tasks derived from the New Yorker Cartoon Caption Contest. We find that both types of models struggle at all three tasks.
  arXiv  Detail & Related papers  (2022-09-13T20:54:00Z)
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding [53.170767750244366]
  Imagen is a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding. To assess text-to-image models in greater depth, we introduce DrawBench, a comprehensive and challenging benchmark for text-to-image models.
  arXiv  Detail & Related papers  (2022-05-23T17:42:53Z)
• CCPM: A Chinese Classical Poetry Matching Dataset [50.90794811956129]
  We propose a novel task to assess a model's semantic understanding of poetry by poem matching. This task requires the model to select one line of Chinese classical poetry among four candidates according to the modern Chinese translation of a line of poetry. To construct this dataset, we first obtain a set of parallel data of Chinese classical poetry and modern Chinese translation.
  arXiv  Detail & Related papers  (2021-06-03T16:49:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.