Re-Centering Humans in LLM Personalization

• URL: http://arxiv.org/abs/2606.06614v1
• Date: Thu, 04 Jun 2026 18:04:47 GMT
• Title: Re-Centering Humans in LLM Personalization
• Abstract summary: We study the gap in personalization performance when using synthetic versus human data.<n>We collect human conversations (550 conversations) and judgments across three stages of personalization.<n>We find that learned reward models achieve only modest correlation with human ratings.
• Score: 15.480006171928435
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data. It remains unclear how well current personalization systems work for real users. In this paper, we study the gap in LLM personalization performance when using synthetic versus human data. We collect human conversations (550 conversations) and judgments across three stages of personalization: extracting user attributes from conversations (5,949 judgments), pairing relevant attributes with new prompts (11,919), and incorporating relevant attributes into a personalized response (1,101). Incorporating human data reveals system limitations at each stage. Models struggle to extract attributes from human conversations, disagree with human judgments on relevant attributes, and generate personalized responses that humans judge no better than generic responses (though that LLM judges widely rate as better). We introduce two lightweight training-based interventions that shift automated personalization evaluation closer to human data in our first two stages. However, in our third stage we find that learned reward models achieve only modest correlation with human ratings, suggesting that human-aligned personalization quality judgments are difficult to model directly. Our collected data provides a foundation for studying how models should extract, select, and incorporate user information in ways that humans find useful.

Related papers

• Evaluating the Hidden Costs of Personalization in Large Language Models [55.07935163900691]
  We identify three emerging risks: (1) irrelevant personalization, where models reference personal information in unnecessary contexts; (2) preference narrowing, where models reinforce informational echo chambers; and (3) sycophantic bias, where models agree excessively with user opinions.<n>We propose PRISK, a dynamic evaluation framework with automated data generation and tailored metrics that uncovers systematic limitations in current AI assistants.
  arXiv  Detail & Related papers  (2026-08-28T20:05:12Z)
• PRISM-X: Experiments on Personalised Fine-Tuning with Human and Simulated Users [14.768125931006798]
  The efficacy of personalisation methods is often evaluated in academic research using simulated users rather than real people.<n>This raises questions about how users and their simulated counterparts differ in interaction patterns and judgements.
  arXiv  Detail & Related papers  (2026-05-13T10:21:14Z)
• ReLay: Personalized LLM-Generated Plain-Language Summaries for Better Understanding, but at What Cost? [54.44351107428039]
  Large language models (LLMs) offer new opportunities for personalizing Plain Language Summaries (PLS)<n>It remains unclear whether personalization helps, which strategies are most effective, and how to balance personalization with safety.<n>We introduce ReLay, a dataset of 300 participant--PLS pairs from 50 lay participants in both static (expert-written) and interactive settings.<n>ReLay includes user characteristics, health information needs, information-seeking behavior, comprehension outcomes, interaction logs, and quality ratings.
  arXiv  Detail & Related papers  (2026-05-01T07:11:12Z)
• HumanLLM: Towards Personalized Understanding and Simulation of Human Nature [72.55730315685837]
  HumanLLM is a foundation model designed for personalized understanding and simulation of individuals.<n>We first construct the Cognitive Genome, a large-scale corpus curated from real-world user data on platforms like Reddit, Twitter, Blogger, and Amazon.<n>We then formulate diverse learning tasks and perform supervised fine-tuning to empower the model to predict a wide range of individualized human behaviors, thoughts, and experiences.
  arXiv  Detail & Related papers  (2026-01-22T09:27:27Z)
• A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations [112.81207927088117]
  PersonaConvBench is a benchmark for evaluating personalized reasoning and generation in multi-turn conversations with large language models (LLMs)<n>We benchmark several commercial and open-source LLMs under a unified prompting setup and observe that incorporating personalized history yields substantial performance improvements.
  arXiv  Detail & Related papers  (2025-05-20T09:13:22Z)
• IMPersona: Evaluating Individual Level LM Impersonation [28.040025302581366]
  We introduce IMPersona, a framework for evaluating LMs at impersonating specific individuals' writing style and personal knowledge.<n>We demonstrate that even modestly sized open-source models, such as Llama-3.1-8B-Instruct, can achieve impersonation abilities at concerning levels.
  arXiv  Detail & Related papers  (2025-04-06T02:57:58Z)
• FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users [111.56469697145519]
  We propose Few-Shot Preference Optimization, which reframes reward modeling as a meta-learning problem.<n>Under this framework, an LLM learns to quickly adapt to a user via a few labeled preferences from that user, constructing a personalized reward function for them.<n>We generate over 1M synthetic personalized preferences using publicly available LLMs.<n>We evaluate FSPO on personalized open-ended generation for up to 1,500 synthetic users across three domains: movie reviews, pedagogical adaptation based on educational background, and general question answering, along with a controlled human study.
  arXiv  Detail & Related papers  (2025-02-26T17:08:46Z)
• Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback [87.37721254914476]
  We introduce HyPER, a Hybrid Preference routER that defers an annotation to either humans or language models (LMs)<n>We show that the selected hybrid mixture of synthetic and direct human preferences using HyPER achieves better RM performance compared to using either one exclusively by 7-13% on RewardBench.<n>We also analyze features from HyPER and find that prompts with moderate safety concerns or complexity benefit the most from human feedback.
  arXiv  Detail & Related papers  (2024-10-24T20:04:15Z)
• Chatting Up Attachment: Using LLMs to Predict Adult Bonds [0.0]
  We use GPT-4 and Claude 3 Opus to create agents that simulate adults with varying profiles, childhood memories, and attachment styles. We evaluate our models using a transcript dataset from 9 humans who underwent the same interview protocol, analyzed and labeled by mental health professionals. Our findings indicate that training the models using only synthetic data achieves performance comparable to training the models on human data.
  arXiv  Detail & Related papers  (2024-08-31T04:29:19Z)
• Personality Alignment of Large Language Models [30.710131188931317]
  Personality Alignment aims to align large language models with individual user preferences.<n>This dataset includes data from over 320,000 real subjects across multiple personality assessments.<n>We develop an activation intervention optimization method to efficiently align with individual behavioral preferences.<n>Our work paves the way for future AI systems to make decisions and reason in truly personality ways.
  arXiv  Detail & Related papers  (2024-08-21T17:09:00Z)
• Comparing Pre-trained Human Language Models: Is it Better with Human Context as Groups, Individual Traits, or Both? [36.03935815294279]
  Human-centered natural language processing requires incorporating human context into language models. Currently, two methods exist: pre-training with 1) group-wise attributes (e.g., over-45-year-olds) or 2) individual traits. We compare pre-training models with human context via 1) group attributes, 2) individual users, and 3) a combined approach on five user- and document-level tasks.
  arXiv  Detail & Related papers  (2024-01-23T05:20:35Z)
• Evaluating Human-Language Model Interaction [79.33022878034627]
  We develop a new framework, Human-AI Language-based Interaction Evaluation (HALIE), that defines the components of interactive systems. We design five tasks to cover different forms of interaction: social dialogue, question answering, crossword puzzles, summarization, and metaphor generation. We find that better non-interactive performance does not always translate to better human-LM interaction.
  arXiv  Detail & Related papers  (2022-12-19T18:59:45Z)
• Improving Personality Consistency in Conversation by Persona Extending [22.124187337032946]
  We propose a novel retrieval-to-prediction paradigm consisting of two subcomponents, namely, Persona Retrieval Model (PRM) and Posterior-scored Transformer (PS-Transformer) Our proposed model yields considerable improvements in both automatic metrics and human evaluations.
  arXiv  Detail & Related papers  (2022-08-23T09:00:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.