Improving Text-to-Image Generation with Input-Side Inference-Time Scaling

• URL: http://arxiv.org/abs/2510.12041v2
• Date: Wed, 15 Oct 2025 03:43:21 GMT
• Title: Improving Text-to-Image Generation with Input-Side Inference-Time Scaling
• Authors: Ruibo Chen, Jiacheng Pan, Heng Huang, Zhenheng Yang,
• Abstract summary: We propose a prompt rewriting framework that leverages large language models to refine user inputs before feeding them into T2I backbones.<n>Results show that our prompt rewriter consistently improves image-text alignment, visual quality, and aesthetics, outperforming strong baselines.<n>These findings highlight that prompt rewriting is an effective, scalable, and practical model-agnostic strategy for improving T2I systems.
• Score: 47.94598818606364
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in text-to-image (T2I) generation have achieved impressive results, yet existing models often struggle with simple or underspecified prompts, leading to suboptimal image-text alignment, aesthetics, and quality. We propose a prompt rewriting framework that leverages large language models (LLMs) to refine user inputs before feeding them into T2I backbones. Our approach introduces a carefully designed reward system and an iterative direct preference optimization (DPO) training pipeline, enabling the rewriter to enhance prompts without requiring supervised fine-tuning data. We evaluate our method across diverse T2I models and benchmarks. Results show that our prompt rewriter consistently improves image-text alignment, visual quality, and aesthetics, outperforming strong baselines. Furthermore, we demonstrate strong transferability by showing that a prompt rewriter trained on one T2I backbone generalizes effectively to others without needing to be retrained. We also systematically study scalability, evaluating how performance gains scale with the capacity of the large LLM used as the rewriter. These findings highlight that prompt rewriting is an effective, scalable, and practical model-agnostic strategy for improving T2I systems. We plan to release the code and trained prompt rewriters soon.

Related papers

• Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization [50.13408999553116]
  We propose RL-Text2Vis, the first reinforcement learning framework for Text2Vis generation.<n>Our method uses a novel multi-objective reward that jointly optimize textual accuracy, code validity, and visualization quality.<n>Our results establish GRPO as an effective strategy for structured, multimodal reasoning in visualization generation.
  arXiv  Detail & Related papers  (2026-01-08T04:29:07Z)
• RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scaling [59.088798018184235]
  textbfRAPO++ is a cross-stage prompt optimization framework.<n>It unifies training-data-aligned refinement, test-time iterative scaling, and large language model fine-tuning.<n> RAPO++ achieves significant gains in semantic alignment, compositional reasoning, temporal stability, and physical plausibility.
  arXiv  Detail & Related papers  (2025-10-23T04:45:09Z)
• AcT2I: Evaluating and Improving Action Depiction in Text-to-Image Models [58.85362281293525]
  We introduce AcT2I, a benchmark designed to evaluate the performance of T2I models in generating images from action-centric prompts.<n>We experimentally validate that leading T2I models do not fare well on AcT2I.<n>We build upon this by developing a training-free, knowledge distillation technique utilizing Large Language Models to address this limitation.
  arXiv  Detail & Related papers  (2025-09-19T16:41:39Z)
• PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting [31.35160142315478]
  We introduce PromptEnhancer, a novel and universal prompt rewriting framework for text-to-image (T2I) models.<n>Unlike prior methods that rely on model-specific fine-tuning or implicit reward signals like image-reward scores, our framework decouples the rewriter from the generator.<n>Experiments on the HunyuanImage 2.1 model demonstrate that PromptEnhancer significantly improves image-text alignment across a wide range of semantic and compositional challenges.
  arXiv  Detail & Related papers  (2025-09-04T16:46:10Z)
• RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning [88.14234949860105]
  RePrompt is a novel reprompting framework that introduces explicit reasoning into the prompt enhancement process via reinforcement learning.<n>Our approach enables end-to-end training without human-annotated data.
  arXiv  Detail & Related papers  (2025-05-23T06:44:26Z)
• Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation [61.31036260686349]
  We propose a novel prompt optimization framework, designed to rephrase a simple user prompt into a sophisticated prompt to a text-to-image model.<n> Specifically, we employ the large vision language models (LVLMs) as the solver to rewrite the user prompt, and concurrently, employ LVLMs as a reward model to score the aesthetics and alignment of the images generated by the optimized prompt.<n>Instead of laborious human feedback, we exploit the prior knowledge of the LVLM to provide rewards, i.e., AI feedback.
  arXiv  Detail & Related papers  (2025-05-22T15:05:07Z)
• ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval [83.01358520910533]
  We introduce a new framework that can boost the performance of large-scale pre-trained vision- curation models.<n>The approach, Enhanced Language-Image Pre-training (ELIP), uses the text query, via a simple mapping network, to predict a set of visual prompts.<n>ELIP can easily be applied to the commonly used CLIP, SigLIP and BLIP-2 networks.
  arXiv  Detail & Related papers  (2025-02-21T18:59:57Z)
• TIPO: Text to Image with Text Presampling for Prompt Optimization [17.312386194139652]
  TIPO (Text-to-Image Prompt Optimization) introduces an efficient approach for automatic prompt refinement in text-to-image (T2I) generation.<n>Starting from simple user prompts, TIPO leverages a lightweight pre-trained model to expand these prompts into richer, detailed versions.
  arXiv  Detail & Related papers  (2024-11-12T19:09:45Z)
• Improving Text-to-Image Consistency via Automatic Prompt Optimization [26.2587505265501]
  We introduce a T2I optimization-by-prompting framework, OPT2I, to improve prompt-image consistency in T2I models. Our framework starts from a user prompt and iteratively generates revised prompts with the goal of maximizing a consistency score.
  arXiv  Detail & Related papers  (2024-03-26T15:42:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.