Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMs

• URL: http://arxiv.org/abs/2603.00024v1
• Date: Tue, 03 Feb 2026 19:04:03 GMT
• Title: Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMs
• Abstract summary: Large Language Models (LLMs) are prone to sycophantic behavior, uncritically conforming to user beliefs.<n>We present a rigorous evaluation of personalization's impact on LLM sycophancy across nine frontier models and five benchmark datasets.
• Score: 2.1198879079315573
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are prone to sycophantic behavior, uncritically conforming to user beliefs. As models increasingly condition responses on user-specific context (personality traits, preferences, conversation history), they gain information to tailor agreement more effectively. Understanding how personalization modulates sycophancy is critical, yet systematic evaluation across models and contexts remains limited. We present a rigorous evaluation of personalization's impact on LLM sycophancy across nine frontier models and five benchmark datasets spanning advice, moral judgment, and debate contexts. We find that personalization generally increases affective alignment (emotional validation, hedging/deference), but affects epistemic alignment (belief adoption, position stability, resistance to influence) with context-dependent role modulation. When the LLM's role is to give advice, personalization strengthens epistemic independence (models challenge user presuppositions). When its role is that of a social peer, personalization decreases epistemic independence. In this role, extensively personalized user challenges causing LLMs to abandon their position at significantly higher rates. Robustness tests confirm that the effects are driven by personalized conditioning, not by additional input tokens per se or demographic information alone. Our work provides measurement frameworks for evaluating personalized AI systems, demonstrates the necessity of role-sensitive evaluation, and establishes a novel benchmark to assess goal alignment.

Related papers

• How Identity and Opinion Shape Political Sycophancy in LLMs [17.0131514909503]
  We introduce a framework that disentangles two distinct triggers of political sycophancy: opinion and identity.<n>Using 450 manually-checked political dilemmas as controlled probes, we evaluate 13 instruction-tuned Large Language Models.<n>Our results suggest that LLM political stance is interactively and steerably vulnerable rather than being a fixed trait.
  arXiv  Detail & Related papers  (2026-08-29T11:12:03Z)
• Evaluating the Hidden Costs of Personalization in Large Language Models [55.07935163900691]
  We identify three emerging risks: (1) irrelevant personalization, where models reference personal information in unnecessary contexts; (2) preference narrowing, where models reinforce informational echo chambers; and (3) sycophantic bias, where models agree excessively with user opinions.<n>We propose PRISK, a dynamic evaluation framework with automated data generation and tailored metrics that uncovers systematic limitations in current AI assistants.
  arXiv  Detail & Related papers  (2026-08-28T20:05:12Z)
• Persuasive and Compliant Tendencies Predict Group Decision-Making in Humans and Language Models [65.21691850622413]
  Large language models (LLMs) are increasingly involved in group decision-making with humans.<n>We introduce DecisionQE, a questionnaire-based framework for measuring each model's persuasive and compliant tendencies.<n>We show that stronger persuasive tendency does not significantly improve group outcomes, whereas compliant-oriented models show more stable advantages in cooperation.
  arXiv  Detail & Related papers  (2026-08-08T15:50:56Z)
• Preference-Aware Rubric Learning for Personalized Evaluation [59.539429430690156]
  Existing evaluation methods fail to capture user-specific preferences embedded in long-term interaction histories.<n>We propose Personalized Evaluation as Learning, a paradigm that formulates personalized evaluation as a learning problem rather than a static judgment.<n>Experiments show that PARL consistently induces high-fidelity rubrics that reliably identify user-aligned responses and generalize across users.
  arXiv  Detail & Related papers  (2026-05-29T17:00:55Z)
• Enhancing Persona Following at Decoding Time via Dynamic Importance Estimation for Role-Playing Agents [13.003892350610947]
  The utility of Role-Playing Language Agents in sociological research is growing alongside the adoption of Large Language Models.<n>For realism in social simulation, Role-Playing Language Agents must adhere to their personas defined by character profiles.<n>We propose a novel, theory-driven method that dynamically estimates context-dependent persona importance and integrates it into weighted reward-guided decoding.
  arXiv  Detail & Related papers  (2026-03-02T04:37:16Z)
• PTCBENCH: Benchmarking Contextual Stability of Personality Traits in LLM Systems [30.449659477704543]
  We introduce PTCBENCH, a benchmark designed to quantify the consistency of large language models (LLMs) personalities under controlled situational contexts.<n> PTCBENCH subjects models to 12 distinct external conditions spanning diverse location contexts and life events, and rigorously assesses the personality using the NEO Five-Factor Inventory.<n>Our study on 39,240 personality trait records reveals that certain external scenarios can trigger significant personality changes of LLMs, and even alter their reasoning capabilities.
  arXiv  Detail & Related papers  (2026-01-12T18:15:50Z)
• RoleRMBench & RoleRM: Towards Reward Modeling for Profile-Based Role Play in Dialogue Systems [85.16327248973387]
  We develop RoleRM, a reward model trained with Continuous Implicit Preferences (CIP)<n>We show RoleRM surpasses strong open- and closed-source reward models by over 24% on average.<n>Our findings highlight the importance of continuous preference representation and annotation consistency, establishing a foundation for subjective alignment in human-centered dialogue systems.
  arXiv  Detail & Related papers  (2025-12-11T12:04:46Z)
• The Personalization Paradox: Semantic Loss vs. Reasoning Gains in Agentic AI Q&A [0.5623023138026949]
  We examined how personalization affects system performance across multiple evaluation dimensions.<n>Results showed a consistent trade-off: personalization reliably improved reasoning quality and grounding.<n>The study demonstrates that personalization produces metric-dependent shifts rather than uniform improvements.
  arXiv  Detail & Related papers  (2025-12-04T00:12:41Z)
• Personalized Reasoning: Just-In-Time Personalization and Why LLMs Fail At It [81.50711040539566]
  Current large language model (LLM) development treats task-solving and preference alignment as separate challenges.<n>We introduce PREFDISCO, an evaluation methodology that transforms static benchmarks into interactive personalization tasks.<n>Our framework creates scenarios where identical questions require different reasoning chains depending on user context.
  arXiv  Detail & Related papers  (2025-09-30T18:55:28Z)
• NextQuill: Causal Preference Modeling for Enhancing LLM Personalization [82.15961484963256]
  We introduce NextQuill, a novel personalization framework grounded in causal preference modeling.<n>Building on this insight, NextQuill introduces two complementary alignment strategies.<n> Experiments across multiple personalization benchmarks demonstrate that NextQuill significantly improves personalization quality.
  arXiv  Detail & Related papers  (2025-06-03T02:08:55Z)
• SOLAR: Towards Characterizing Subjectivity of Individuals through Modeling Value Conflicts and Trade-offs [19.764066916148007]
  We characterize subjectivity of individuals on social media and infer their moral judgments using Large Language Models.<n>We propose a framework, SOLAR, that observes value conflicts and trade-offs in the user-generated texts to better represent subjective ground of individuals.
  arXiv  Detail & Related papers  (2025-04-17T04:20:05Z)
• Personality Editing for Language Models through Adjusting Self-Referential Queries [17.051166122108857]
  We present PALETTE (Personality Adjustment by LLM SElf-TargeTed quEries), a novel method for personality editing in Large Language Models (LLMs)<n>Our approach introduces adjustment queries, where self-referential statements grounded in psychological constructs are treated analogously to factual knowledge, enabling direct editing of personality-related responses.<n>Unlike fine-tuning, PALETTE requires only 12 editing samples to achieve substantial improvements in personality alignment across personality dimensions.
  arXiv  Detail & Related papers  (2025-02-17T13:28:14Z)
• Self-Evolving Critique Abilities in Large Language Models [59.861013614500024]
  This paper explores enhancing critique abilities of Large Language Models (LLMs)<n>We introduce SCRIT, a framework that trains LLMs with self-generated data to evolve their critique abilities.<n>Our analysis reveals that SCRIT's performance scales positively with data and model size.
  arXiv  Detail & Related papers  (2025-01-10T05:51:52Z)
• Evaluating Large Language Models with Psychometrics [59.821829073478376]
  This paper offers a comprehensive benchmark for quantifying psychological constructs of Large Language Models (LLMs)<n>Our work identifies five key psychological constructs -- personality, values, emotional intelligence, theory of mind, and self-efficacy -- assessed through a suite of 13 datasets.<n>We uncover significant discrepancies between LLMs' self-reported traits and their response patterns in real-world scenarios, revealing complexities in their behaviors.
  arXiv  Detail & Related papers  (2024-06-25T16:09:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.