Personalized Reward Modeling for Text-to-Image Generation

• URL: http://arxiv.org/abs/2511.19458v1
• Date: Fri, 21 Nov 2025 12:04:24 GMT
• Title: Personalized Reward Modeling for Text-to-Image Generation
• Authors: Jeongeun Lee, Ryang Heo, Dongha Lee,
• Abstract summary: We present PIGReward, a personalized reward model that dynamically generates user-conditioned evaluation dimensions and assesses images through CoT reasoning.<n> PIGReward provides personalized feedback that drives user-specific prompt optimization, improving alignment between generated images and individual intent.<n>Extensive experiments demonstrate that PIGReward surpasses existing methods in both accuracy and interpretability.
• Score: 9.780251969338044
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent text-to-image (T2I) models generate semantically coherent images from textual prompts, yet evaluating how well they align with individual user preferences remains an open challenge. Conventional evaluation methods, general reward functions or similarity-based metrics, fail to capture the diversity and complexity of personal visual tastes. In this work, we present PIGReward, a personalized reward model that dynamically generates user-conditioned evaluation dimensions and assesses images through CoT reasoning. To address the scarcity of user data, PIGReward adopt a self-bootstrapping strategy that reasons over limited reference data to construct rich user contexts, enabling personalization without user-specific training. Beyond evaluation, PIGReward provides personalized feedback that drives user-specific prompt optimization, improving alignment between generated images and individual intent. We further introduce PIGBench, a per-user preference benchmark capturing diverse visual interpretations of shared prompts. Extensive experiments demonstrate that PIGReward surpasses existing methods in both accuracy and interpretability, establishing a scalable and reasoning-based foundation for personalized T2I evaluation and optimization. Taken together, our findings highlight PIGReward as a robust steptoward individually aligned T2I generation.

Related papers

• P-GenRM: Personalized Generative Reward Model with Test-time User-based Scaling [66.55381105691818]
  We propose P-GenRM, the first Personalized Generative Reward Model with test-time user-based scaling.<n>P-GenRM transforms preference signals into structured evaluation chains that derive adaptive personas and scoring rubrics.<n>It further clusters users into User Prototypes and introduces a dual-granularity scaling mechanism.
  arXiv  Detail & Related papers  (2026-02-12T16:07:22Z)
• PreferThinker: Reasoning-based Personalized Image Preference Assessment [83.66114370585976]
  We propose a reasoning-based personalized image preference assessment framework.<n>It first predicts a user's preference profile from reference images.<n>It then provides interpretable, multi-dimensional scores and assessments of candidate images.
  arXiv  Detail & Related papers  (2025-11-01T16:19:51Z)
• Personalize Before Retrieve: LLM-based Personalized Query Expansion for User-Centric Retrieval [34.298743064665395]
  Personalize Before Retrieve (PBR) is a framework that incorporates user-specific signals into query expansion prior to retrieval.<n>PBR consistently outperforms strong baselines, with up to 10% gains on PersonaBench across retrievers.
  arXiv  Detail & Related papers  (2025-10-10T02:24:09Z)
• Improving Personalized Search with Regularized Low-Rank Parameter Updates [52.29168893900888]
  We show how to adapt the internal representation of a vision-language dual encoder model for personalized vision-language retrieval.<n>We find that regularized low-rank adaption of a small set of parameters in the language encoder's final layer serves as a highly effective alternative to textual inversion.<n>Our approach achieves state-of-the-art accuracy on two benchmarks for personalized image retrieval with natural language queries.
  arXiv  Detail & Related papers  (2025-06-11T21:15:21Z)
• RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning [88.14234949860105]
  RePrompt is a novel reprompting framework that introduces explicit reasoning into the prompt enhancement process via reinforcement learning.<n>Our approach enables end-to-end training without human-annotated data.
  arXiv  Detail & Related papers  (2025-05-23T06:44:26Z)
• RAGAR: Retrieval Augmented Personalized Image Generation Guided by Recommendation [15.042729128202982]
  We propose Retrieval Augment Personalized Image GenerAtion guided by Recommendation (RAGAR)<n>Our approach uses a retrieval mechanism to assign different weights to historical items according to their similarities to the reference item.<n> RAGAR achieves significant improvements in both personalization and semantic metrics compared to five baselines.
  arXiv  Detail & Related papers  (2025-05-03T02:20:30Z)
• DRC: Enhancing Personalized Image Generation via Disentangled Representation Composition [69.10628479553709]
  We introduce DRC, a novel personalized image generation framework that enhances Large Multimodal Models (LMMs)<n> DRC explicitly extracts user style preferences and semantic intentions from history images and the reference image, respectively.<n>It involves two critical learning stages: 1) Disentanglement learning, which employs a dual-tower disentangler to explicitly separate style and semantic features, optimized via a reconstruction-driven paradigm with difficulty-aware importance sampling; and 2) Personalized modeling, which applies semantic-preserving augmentations to effectively adapt the disentangled representations for robust personalized generation.
  arXiv  Detail & Related papers  (2025-04-24T08:10:10Z)
• Personalized Graph-Based Retrieval for Large Language Models [51.7278897841697]
  We propose a framework that leverages user-centric knowledge graphs to enrich personalization.<n>By directly integrating structured user knowledge into the retrieval process and augmenting prompts with user-relevant context, PGraph enhances contextual understanding and output quality.<n>We also introduce the Personalized Graph-based Benchmark for Text Generation, designed to evaluate personalized text generation tasks in real-world settings where user history is sparse or unavailable.
  arXiv  Detail & Related papers  (2025-01-04T01:46:49Z)
• Preference Adaptive and Sequential Text-to-Image Generation [24.787970969428976]
  We create a novel dataset of sequential preferences, which we leverage, together with large-scale open-source (non-sequential) datasets.<n>We construct user-preference and user-choice models using an EM strategy and identify varying user preference types.<n>We then leverage a large multimodal language model (LMM) and a value-based RL approach to suggest an adaptive and diverse slate of prompt expansions to the user.<n>Our Preference Adaptive and Sequential Text-to-image Agent (PASTA) extends T2I models with adaptive multi-turn capabilities, fostering collaborative co-creation and addressing uncertainty or underspecification
  arXiv  Detail & Related papers  (2024-12-10T01:47:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.