WikiPersonas: What Can We Learn From Personalized Alignment to Famous People?

• URL: http://arxiv.org/abs/2505.13257v1
• Date: Mon, 19 May 2025 15:39:48 GMT
• Title: WikiPersonas: What Can We Learn From Personalized Alignment to Famous People?
• Authors: Zilu Tang, Afra Feyza Akyürek, Ekin Akyürek, Derry Wijaya,
• Abstract summary: We introduce WikiPersona: the first fine-grained personalization using well-documented, famous individuals.<n>We evaluate different personalization approaches and find that using textitinferred personal preferences as prefixes enables effective personalization.
• Score: 14.801237597577169
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Preference alignment has become a standard pipeline in finetuning models to follow \emph{generic} human preferences. Majority of work seeks to optimize model to produce responses that would be preferable \emph{on average}, simplifying the diverse and often \emph{contradicting} space of human preferences. While research has increasingly focused on personalized alignment: adapting models to individual user preferences, there is a lack of personalized preference dataset which focus on nuanced individual-level preferences. To address this, we introduce WikiPersona: the first fine-grained personalization using well-documented, famous individuals. Our dataset challenges models to align with these personas through an interpretable process: generating verifiable textual descriptions of a persona's background and preferences in addition to alignment. We systematically evaluate different personalization approaches and find that as few-shot prompting with preferences and fine-tuning fail to simultaneously ensure effectiveness and efficiency, using \textit{inferred personal preferences} as prefixes enables effective personalization, especially in topics where preferences clash while leading to more equitable generalization across unseen personas.

Related papers

• PrefPalette: Personalized Preference Modeling with Latent Attributes [59.58648056175468]
  PrefPalette is a framework that decomposes preferences into attribute dimensions.<n>It tailors its preference prediction to distinct social community values.<n>PrefPalette outperforms GPT-4o by 46.6% in average prediction accuracy.
  arXiv  Detail & Related papers  (2025-07-17T21:21:54Z)
• NextQuill: Causal Preference Modeling for Enhancing LLM Personalization [82.15961484963256]
  We introduce NextQuill, a novel personalization framework grounded in causal preference modeling.<n>Building on this insight, NextQuill introduces two complementary alignment strategies.<n> Experiments across multiple personalization benchmarks demonstrate that NextQuill significantly improves personalization quality.
  arXiv  Detail & Related papers  (2025-06-03T02:08:55Z)
• Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment [21.677859755364334]
  Persona-judge is a novel discriminative paradigm that enables training-free personalized alignment with unseen preferences.<n>We show that Persona-judge offers a scalable and computationally efficient solution to personalized alignment.
  arXiv  Detail & Related papers  (2025-04-17T05:50:13Z)
• From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment [41.96246165999026]
  Large language models (LLMs) have traditionally been aligned through one-size-fits-all approaches.<n>This paper introduces a comprehensive framework for scalable personalized alignment of LLMs.
  arXiv  Detail & Related papers  (2025-03-19T17:41:46Z)
• Personalized Preference Fine-tuning of Diffusion Models [75.22218338096316]
  We introduce PPD, a multi-reward optimization objective that aligns diffusion models with personalized preferences.<n>With PPD, a diffusion model learns the individual preferences of a population of users in a few-shot way.<n>Our approach achieves an average win rate of 76% over Stable Cascade, generating images that more accurately reflect specific user preferences.
  arXiv  Detail & Related papers  (2025-01-11T22:38:41Z)
• ComPO: Community Preferences for Language Model Personalization [122.54846260663922]
  ComPO is a method to personalize preference optimization in language models. We collect and release ComPRed, a question answering dataset with community-level preferences from Reddit.
  arXiv  Detail & Related papers  (2024-10-21T14:02:40Z)
• PREDICT: Preference Reasoning by Evaluating Decomposed preferences Inferred from Candidate Trajectories [3.0102456679931944]
  This paper introduces PREDICT, a method designed to enhance the precision and adaptability of inferring preferences. We evaluate PREDICT on two distinct environments: a gridworld setting and a new text-domain environment.
  arXiv  Detail & Related papers  (2024-10-08T18:16:41Z)
• Unsupervised Human Preference Learning [7.959043497459107]
  Large language models demonstrate impressive reasoning abilities but struggle to provide personalized content. Existing methods, such as in-context learning and parameter-efficient fine-tuning, fall short in capturing the complexity of human preferences. We propose a novel approach utilizing small parameter models as preference agents to generate natural language rules that guide a larger, pre-trained model.
  arXiv  Detail & Related papers  (2024-09-30T17:51:01Z)
• DegustaBot: Zero-Shot Visual Preference Estimation for Personalized Multi-Object Rearrangement [53.86523017756224]
  We present DegustaBot, an algorithm for visual preference learning that solves household multi-object rearrangement tasks according to personal preference. We collect a large dataset of naturalistic personal preferences in a simulated table-setting task. We find that 50% of our model's predictions are likely to be found acceptable by at least 20% of people.
  arXiv  Detail & Related papers  (2024-07-11T21:28:02Z)
• Personalized Language Modeling from Personalized Human Feedback [45.16986573937782]
  Personalized large language models (LLMs) are designed to tailor responses to individual user preferences.<n>We propose Personalized-RLHF, an efficient framework that utilizes a lightweight user model to capture individual user preferences.<n>We show that personalized LLMs trained using P-RLHF generate responses that are more closely aligned with individual user preferences.
  arXiv  Detail & Related papers  (2024-02-06T04:18:58Z)
• Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging [148.77027765872006]
  We study Reinforcement Learning from Personalized Human Feedback (RLPHF) problem. LLMs are aligned to multiple preferences by modeling alignment as a Multi-Objective Reinforcement Learning (MORL) problem. We show that we can achieve personalized alignment by decomposing preferences into multiple dimensions.
  arXiv  Detail & Related papers  (2023-10-17T20:22:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.