PromptEcho: Annotation-Free Reward from Vision-Language Models for Text-to-Image Reinforcement Learning

• URL: http://arxiv.org/abs/2604.12652v2
• Date: Wed, 22 Apr 2026 08:05:01 GMT
• Title: PromptEcho: Annotation-Free Reward from Vision-Language Models for Text-to-Image Reinforcement Learning
• Abstract summary: Reinforcement learning can improve the prompt following capability of text-to-image (T2I) models.<n>We propose PromptEcho, a reward construction method that requires emphno annotation and emphno reward model training.
• Score: 14.637088591693088
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning (RL) can improve the prompt following capability of text-to-image (T2I) models, yet obtaining high-quality reward signals remains challenging: CLIP Score is too coarse-grained, while VLM-based reward models (e.g., RewardDance) require costly human-annotated preference data and additional fine-tuning. We propose PromptEcho, a reward construction method that requires \emph{no} annotation and \emph{no} reward model training. Given a generated image and a guiding query, PromptEcho computes the token-level cross-entropy loss of a frozen VLM with the original prompt as the label, directly extracting the image-text alignment knowledge encoded during VLM pretraining. The reward is deterministic, computationally efficient, and improves automatically as stronger open-source VLMs become available. For evaluation, we develop DenseAlignBench, a benchmark of concept-rich dense captions for rigorously testing prompt following capability. Experimental results on two state-of-the-art T2I models (Z-Image and QwenImage-2512) demonstrate that PromptEcho achieves substantial improvements on DenseAlignBench (+26.8pp / +16.2pp net win rate), along with consistent gains on GenEval, DPG-Bench, and TIIFBench without any task-specific training. Ablation studies confirm that PromptEcho comprehensively outperforms inference-based scoring with the same VLM, and that reward quality scales with VLM size. We will open-source the trained models and the DenseAlignBench.

Related papers

• Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation [52.00324244076857]
  SpectraReward turns pretrained MLLMs into off-the-shelf reward models for image-generation reinforcement learning.<n>Self-SpectraReward forms a closed-loop self-improving framework without external reward models or external knowledge.
  arXiv  Detail & Related papers  (2026-07-13T17:59:58Z)
• Qwen-Image-2.0-RL Technical Report [66.2346321737379]
  We present Qwen-Image-2.0-RL, a post-training pipeline that applies reinforcement learning from human feedback and on-policy distillation.<n>Qwen-Image-2.0-RL achieves 57.84 overall score on Qwen-Image-Bench, Elo ratings of 1193 in text-to-image arena and 1349 in image edit arena.
  arXiv  Detail & Related papers  (2026-06-25T23:49:50Z)
• Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards [52.42920996842378]
  Most unified large multimodal models (LMMs) that support both visual understanding and image generation rely on curated post-training supervision.<n>We propose a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a solver that answers and evaluates them, and a Generator that synthesizes images.<n>For image generation, we design a multi-scale internal evaluation scheme that combines question-answer fidelity scoring with cycle-consistent captioning.
  arXiv  Detail & Related papers  (2026-06-25T17:59:57Z)
• Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing [52.825281124618535]
  Reaction diagram parsing (RxnDP) is critical for extracting chemical synthesis information from literature.<n>Recent Vision-Language Models (VLMs) have emerged as a promising paradigm to automate this complex visual reasoning task.<n>This work enhances VLM-based RxnDP from two complementary perspectives: prompting representation and learning paradigms.
  arXiv  Detail & Related papers  (2026-03-16T09:17:05Z)
• PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting [31.35160142315478]
  We introduce PromptEnhancer, a novel and universal prompt rewriting framework for text-to-image (T2I) models.<n>Unlike prior methods that rely on model-specific fine-tuning or implicit reward signals like image-reward scores, our framework decouples the rewriter from the generator.<n>Experiments on the HunyuanImage 2.1 model demonstrate that PromptEnhancer significantly improves image-text alignment across a wide range of semantic and compositional challenges.
  arXiv  Detail & Related papers  (2025-09-04T16:46:10Z)
• Free$^2$Guide: Training-Free Text-to-Video Alignment using Image LVLM [54.755023509791364]
  Free$2$Guide is a gradient-free and training-free framework for aligning generated videos with text prompts.<n>Our framework supports the flexible ensembling of multiple reward models to enhance alignment without significant computational overhead.
  arXiv  Detail & Related papers  (2024-11-26T02:14:47Z)
• Learning Prompt with Distribution-Based Feature Replay for Few-Shot Class-Incremental Learning [56.29097276129473]
  We propose a simple yet effective framework, named Learning Prompt with Distribution-based Feature Replay (LP-DiF)<n>To prevent the learnable prompt from forgetting old knowledge in the new session, we propose a pseudo-feature replay approach.<n>When progressing to a new session, pseudo-features are sampled from old-class distributions combined with training images of the current session to optimize the prompt.
  arXiv  Detail & Related papers  (2024-01-03T07:59:17Z)
• Prompt-based Learning for Unpaired Image Captioning [86.44188293709307]
  Unpaired Image Captioning (UIC) has been developed to learn image descriptions from unaligned vision-language sample pairs. Recent successes of Vision-Language Pre-Trained Models (VL-PTMs) have triggered the development of prompt-based learning. We present in this paper a novel scheme based on prompt to train the UIC model, making best use of the powerful generalization ability.
  arXiv  Detail & Related papers  (2022-05-26T03:13:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.