Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue

• URL: http://arxiv.org/abs/2609.12373v1
• Date: Fri, 11 Sep 2026 02:43:09 GMT
• Title: Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue
• Abstract summary: Persona drift remains a central challenge for personalized language models.<n>We propose CORE, which separates turn-local evidence from persistent persona-state revision.<n>We also introduce PERSIST, a benchmark for persona-state robustness under sequential interaction stress.
• Score: 21.86720581525853
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Persona drift remains a central challenge for personalized language models, as user profiles evolve over long interactions rather than remain permanently fixed. Models must therefore revise persistent persona states when preferences genuinely change, while avoiding updates driven by transient, ambiguous, or unresolved observations. We propose CORE, which separates turn-local evidence from persistent persona-state revision and selectively updates grounded user preferences through uncertainty-aware belief revision. We also introduce PERSIST, a held-out post-anchor benchmark for persona-state robustness under sequential interaction stress, covering ambiguity, conflict, and controlled social influence. Across ALOE, PersonaChat, and PERSIST, CORE improves personalized alignment and robustness, with complementary gains in normalized closed-slot state fidelity. Human evaluation and mechanistic controls further support explicit update control beyond stronger generation or persistent memory alone.

Related papers

• Hypotheses-Guided Self Distillation for Continual Personalization [16.17908994206264]
  We introduce HypReflect, a reliable, scalable framework for continual personalization.<n>HypReflect infers explicit, uncertainty-aware preference hypotheses from diverse user signals.<n>Results suggest a step towards reliable and scalable continual personalization.
  arXiv  Detail & Related papers  (2026-08-31T18:52:21Z)
• From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents [56.187590188698955]
  Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful.<n>Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles.<n>We introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories.<n>We propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment.
  arXiv  Detail & Related papers  (2026-08-03T12:52:04Z)
• Preference-Aware Rubric Learning for Personalized Evaluation [59.539429430690156]
  Existing evaluation methods fail to capture user-specific preferences embedded in long-term interaction histories.<n>We propose Personalized Evaluation as Learning, a paradigm that formulates personalized evaluation as a learning problem rather than a static judgment.<n>Experiments show that PARL consistently induces high-fidelity rubrics that reliably identify user-aligned responses and generalize across users.
  arXiv  Detail & Related papers  (2026-05-29T17:00:55Z)
• VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions [63.13827503828231]
  We introduce VitaBench 2.0, a benchmark for evaluating personalized and proactive agent behavior in long-term user interactions.<n>Results show that real-world personalization remains highly challenging even for state-of-the-art models.
  arXiv  Detail & Related papers  (2026-05-26T15:07:38Z)
• PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments [72.02445514666428]
  We introduce PERMA, a benchmark designed to evaluate persona consistency over time beyond static preference recall.<n>PerMA consists of temporally ordered interaction events spanning multiple sessions and domains, with preference-related queries inserted over time.<n> Experiments demonstrate that by linking related interactions, advanced memory systems can extract more precise preferences and reduce token consumption.
  arXiv  Detail & Related papers  (2026-03-24T14:04:11Z)
• Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMs [2.1198879079315573]
  Large Language Models (LLMs) are prone to sycophantic behavior, uncritically conforming to user beliefs.<n>We present a rigorous evaluation of personalization's impact on LLM sycophancy across nine frontier models and five benchmark datasets.
  arXiv  Detail & Related papers  (2026-02-03T19:04:03Z)
• SPRInG: Continual LLM Personalization via Selective Parametric Adaptation and Retrieval-Interpolated Generation [11.253466187411524]
  We introduce SPRInG, a novel semi-parametric framework designed for effective continual personalization.<n>During training, SPRInG employs drift-driven selective adaptation, which utilizes a likelihood-based scoring function to identify high-novelty interactions.<n>Experiments on the long-form personalized generation benchmark demonstrate that SPRInG outperforms existing baselines.
  arXiv  Detail & Related papers  (2026-01-15T01:32:27Z)
• PersonaDual: Balancing Personalization and Objectivity via Adaptive Reasoning [57.10486355722133]
  PersonaDual supports both general-purpose objective reasoning and personalized reasoning in a single model.<n>Experiments on objective and personalized benchmarks show that PersonaDual preserves the benefits of personalization while reducing interference.
  arXiv  Detail & Related papers  (2026-01-13T16:02:35Z)
• Towards Proactive Personalization through Profile Customization for Individual Users in Dialogues [28.522406727886395]
  PersonalAgent is a lifelong agent designed to continuously infer and adapt to user preferences.<n>Experiments show that PersonalAgent achieves superior performance over strong prompt-based and policy optimization baselines.<n>Our findings underscore the importance of lifelong personalization for developing more inclusive and adaptive conversational agents.
  arXiv  Detail & Related papers  (2025-12-17T10:47:06Z)
• Personalized Reasoning: Just-In-Time Personalization and Why LLMs Fail At It [81.50711040539566]
  Current large language model (LLM) development treats task-solving and preference alignment as separate challenges.<n>We introduce PREFDISCO, an evaluation methodology that transforms static benchmarks into interactive personalization tasks.<n>Our framework creates scenarios where identical questions require different reasoning chains depending on user context.
  arXiv  Detail & Related papers  (2025-09-30T18:55:28Z)
• Spotting Out-of-Character Behavior: Atomic-Level Evaluation of Persona Fidelity in Open-Ended Generation [16.76995815742803]
  We propose an atomic-level evaluation framework that quantifies persona fidelity at a finer granularity.<n>Our three key metrics measure the degree of persona alignment and consistency within and across generations.<n>By analyzing persona fidelity across diverse tasks and personality types, we reveal how task structure and persona desirability influence model adaptability.
  arXiv  Detail & Related papers  (2025-06-24T06:33:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.