Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation

• URL: http://arxiv.org/abs/2603.20725v1
• Date: Sat, 21 Mar 2026 09:19:12 GMT
• Title: Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation
• Abstract summary: We present Premier, a novel preference modulation framework for personalized image generation.<n>Premier represents each user's preference as a learnable embedding and introduces a preference adapter that fuses the user embedding with the text prompt.<n>Experiments show that Premier outperforms prior methods under the same history length.
• Score: 67.23759365022279
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-image generation has advanced rapidly, yet it still struggles to capture the nuanced user preferences. Existing approaches typically rely on multimodal large language models to infer user preferences, but the derived prompts or latent codes rarely reflect them faithfully, leading to suboptimal personalization. We present Premier, a novel preference modulation framework for personalized image generation. Premier represents each user's preference as a learnable embedding and introduces a preference adapter that fuses the user embedding with the text prompt. To enable accurate and fine-grained preference control, the fused preference embedding is further used to modulate the generative process. To enhance the distinctness of individual preference and improve alignment between outputs and user-specific styles, we incorporate a dispersion loss that enforces separation among user embeddings. When user data are scarce, new users are represented as linear combinations of existing preference embeddings learned during training, enabling effective generalization. Experiments show that Premier outperforms prior methods under the same history length, achieving stronger preference alignment and superior performance on text consistency, ViPer proxy metrics, and expert evaluations.

Related papers

• PrefReward: Learning User Preference Matrix for Personalized Text Generation [41.05380951849894]
  We propose PrefReward, a preference-aware generative framework for personalization.<n>PrefReward explicitly models user styles through a structured preference matrix and integrates it into the decoding process as a reward signal.<n> Experiments on the LongLaMP dataset show that PrefReward outperforms non-personalized and retrieval-based baselines in both generation quality and personalization interpretability.
  arXiv  Detail & Related papers  (2026-07-23T09:00:20Z)
• Synthetic Interaction Data for Scalable Personalization in Large Language Models [67.31884245564086]
  We introduce a high-fidelity synthetic data generation framework called PersonaGym.<n>Unlike prior work that treats personalization as static persona-preference pairs, PersonaGym models a dynamic preference process.<n>We release PersonaAtlas, a large-scale, high-quality, and diverse synthetic dataset of high-fidelity multi-turn personalized interaction trajectories.
  arXiv  Detail & Related papers  (2026-02-12T20:41:22Z)
• EXACT: Explicit Attribute-Guided Decoding-Time Personalization [11.035465374731563]
  EXACT is a new decoding-time personalization that aligns generation with limited pairwise preference feedback.<n>We show that EXACT consistently outperforms strong baselines, including preference modeling accuracy and personalized generation quality.
  arXiv  Detail & Related papers  (2026-02-06T14:53:37Z)
• Reasoning-Based Personalized Generation for Users with Sparse Data [120.94029850012045]
  We introduce GraSPer, a novel framework for enhancing personalized text generation under sparse context.<n>GraSPer first augments user context by predicting items that the user would likely interact with in the future.<n>With reasoning alignment, it then generates texts for these interactions to enrich the augmented context.<n>In the end, it generates personalized outputs conditioned on both the real and synthetic histories.
  arXiv  Detail & Related papers  (2026-01-31T01:54:23Z)
• PrefGen: Multimodal Preference Learning for Preference-Conditioned Image Generation [44.638427866399205]
  Preference-conditioned image generation seeks to adapt generative models to individual users, producing outputs that reflect personal aesthetic choices beyond the given prompt.<n>We propose a framework that leverages multimodal large language models to extract rich user representations and inject them into diffusion-based image generation.<n>Our method substantially outperforms strong baselines in both image quality and preference alignment.
  arXiv  Detail & Related papers  (2025-12-04T02:57:29Z)
• PreferThinker: Reasoning-based Personalized Image Preference Assessment [83.66114370585976]
  We propose a reasoning-based personalized image preference assessment framework.<n>It first predicts a user's preference profile from reference images.<n>It then provides interpretable, multi-dimensional scores and assessments of candidate images.
  arXiv  Detail & Related papers  (2025-11-01T16:19:51Z)
• Learning User Preferences for Image Generation Model [15.884017849539754]
  We propose an approach built upon Multimodal Large Language Models to learn personalized user preferences.<n>The contrastive preference loss is designed to effectively distinguish between user ''likes'' and ''dislikes''<n>The learnable preference tokens capture shared interest representations among existing users, enabling the model to activate group-specific preferences and enhance consistency across similar users.
  arXiv  Detail & Related papers  (2025-08-11T17:39:42Z)
• Towards Explainable Temporal User Profiling with LLMs [3.719862246745416]
  We leverage large language models (LLMs) to generate natural language summaries of users' interaction histories.<n>Our framework not only models temporal user preferences but also produces natural language profiles that can be used to explain recommendations in an interpretable manner.
  arXiv  Detail & Related papers  (2025-05-01T22:02:46Z)
• Personalized Preference Fine-tuning of Diffusion Models [75.22218338096316]
  We introduce PPD, a multi-reward optimization objective that aligns diffusion models with personalized preferences.<n>With PPD, a diffusion model learns the individual preferences of a population of users in a few-shot way.<n>Our approach achieves an average win rate of 76% over Stable Cascade, generating images that more accurately reflect specific user preferences.
  arXiv  Detail & Related papers  (2025-01-11T22:38:41Z)
• Aligning LLM Agents by Learning Latent Preference from User Edits [23.235995078727658]
  We study interactive learning of language agents based on user edits made to the agent's output. We propose a learning framework, PRELUDE, that infers a description of the user's latent preference based on historic edit data. We introduce two interactive environments -- summarization and email writing, and use a GPT-4 simulated user for evaluation.
  arXiv  Detail & Related papers  (2024-04-23T17:57:47Z)
• Modeling Dynamic User Preference via Dictionary Learning for Sequential Recommendation [133.8758914874593]
  Capturing the dynamics in user preference is crucial to better predict user future behaviors because user preferences often drift over time. Many existing recommendation algorithms -- including both shallow and deep ones -- often model such dynamics independently. This paper considers the problem of embedding a user's sequential behavior into the latent space of user preferences.
  arXiv  Detail & Related papers  (2022-04-02T03:23:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.