PersonaDual: Balancing Personalization and Objectivity via Adaptive Reasoning

• URL: http://arxiv.org/abs/2601.08679v1
• Date: Tue, 13 Jan 2026 16:02:35 GMT
• Title: PersonaDual: Balancing Personalization and Objectivity via Adaptive Reasoning
• Authors: Xiaoyou Liu, Xinyi Mou, Shengbin Yue, Liang Wang, Yuqing Wang, Qiexiang Wang, Tianrui Qin, Wangchunshu Zhou, Zhongyu Wei,
• Abstract summary: PersonaDual supports both general-purpose objective reasoning and personalized reasoning in a single model.<n>Experiments on objective and personalized benchmarks show that PersonaDual preserves the benefits of personalization while reducing interference.
• Score: 57.10486355722133
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As users increasingly expect LLMs to align with their preferences, personalized information becomes valuable. However, personalized information can be a double-edged sword: it can improve interaction but may compromise objectivity and factual correctness, especially when it is misaligned with the question. To alleviate this problem, we propose PersonaDual, a framework that supports both general-purpose objective reasoning and personalized reasoning in a single model, and adaptively switches modes based on context. PersonaDual is first trained with SFT to learn two reasoning patterns, and then further optimized via reinforcement learning with our proposed DualGRPO to improve mode selection. Experiments on objective and personalized benchmarks show that PersonaDual preserves the benefits of personalization while reducing interference, achieving near interference-free performance and better leveraging helpful personalized signals to improve objective problem-solving.

Related papers

• Synthetic Interaction Data for Scalable Personalization in Large Language Models [67.31884245564086]
  We introduce a high-fidelity synthetic data generation framework called PersonaGym.<n>Unlike prior work that treats personalization as static persona-preference pairs, PersonaGym models a dynamic preference process.<n>We release PersonaAtlas, a large-scale, high-quality, and diverse synthetic dataset of high-fidelity multi-turn personalized interaction trajectories.
  arXiv  Detail & Related papers  (2026-02-12T20:41:22Z)
• One Adapts to Any: Meta Reward Modeling for Personalized LLM Alignment [55.86333374784959]
  We argue that addressing these constraints requires a paradigm shift from fitting data to learn user preferences to learn the process of preference adaptation.<n>We propose Meta Reward Modeling (MRM), which reformulates personalized reward modeling as a meta-learning problem.<n>We show that MRM enhances few-shot personalization, improves user robustness, and consistently outperforms baselines.
  arXiv  Detail & Related papers  (2026-01-26T17:55:52Z)
• When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMs [13.695058536403108]
  We show that when personalized large language models (LLMs) face factual queries, the model generates answers aligned with a user's prior history rather than the objective truth.<n>We propose Factuality-Preserving Personalized Steering (FPPS), a lightweight inference-time approach that mitigates personalization-induced factual distortions.
  arXiv  Detail & Related papers  (2026-01-16T05:20:10Z)
• Towards Faithful and Controllable Personalization via Critique-Post-Edit Reinforcement Learning [22.252030067675065]
  We propose Critique-Post-Edit, a robust reinforcement learning framework that enables more faithful and controllable personalization.<n>Our framework integrates two key components: (1) a Personalized Generative Reward Model (GRM) that provides multi-dimensional scores and textual critiques to resist reward hacking, and (2) a Critique-Post-Edit mechanism where the policy model revises its own outputs based on these critiques for more targeted and efficient learning.
  arXiv  Detail & Related papers  (2025-10-21T17:40:03Z)
• Personalized Reasoning: Just-In-Time Personalization and Why LLMs Fail At It [81.50711040539566]
  Current large language model (LLM) development treats task-solving and preference alignment as separate challenges.<n>We introduce PREFDISCO, an evaluation methodology that transforms static benchmarks into interactive personalization tasks.<n>Our framework creates scenarios where identical questions require different reasoning chains depending on user context.
  arXiv  Detail & Related papers  (2025-09-30T18:55:28Z)
• NextQuill: Causal Preference Modeling for Enhancing LLM Personalization [82.15961484963256]
  We introduce NextQuill, a novel personalization framework grounded in causal preference modeling.<n>Building on this insight, NextQuill introduces two complementary alignment strategies.<n> Experiments across multiple personalization benchmarks demonstrate that NextQuill significantly improves personalization quality.
  arXiv  Detail & Related papers  (2025-06-03T02:08:55Z)
• Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward [15.054368169735872]
  We propose leveraging a user model to incorporate a curiosity-based intrinsic reward into multi-turn RLHF.<n>This novel reward mechanism encourages the LLM agent to actively infer user traits by optimizing conversations to improve its user model's accuracy.<n>We demonstrate our method's effectiveness in two distinct domains: significantly improving personalization performance in a conversational recommendation task, and personalizing conversations for different learning styles in an educational setting.
  arXiv  Detail & Related papers  (2025-04-04T06:35:02Z)
• Personalized Adaptation via In-Context Preference Learning [20.042909385219716]
  Preference Pretrained Transformer (PPT) is a novel approach for adaptive personalization using online user feedback. Our results suggest the potential of in-context learning for scalable and efficient personalization in large language models.
  arXiv  Detail & Related papers  (2024-10-17T20:06:02Z)
• Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment [103.12563033438715]
  Alignment in artificial intelligence pursues consistency between model responses and human preferences as well as values. Existing alignment techniques are mostly unidirectional, leading to suboptimal trade-offs and poor flexibility over various objectives. We introduce controllable preference optimization (CPO), which explicitly specifies preference scores for different objectives.
  arXiv  Detail & Related papers  (2024-02-29T12:12:30Z)
• Personalized Language Modeling from Personalized Human Feedback [45.16986573937782]
  Personalized large language models (LLMs) are designed to tailor responses to individual user preferences.<n>We propose Personalized-RLHF, an efficient framework that utilizes a lightweight user model to capture individual user preferences.<n>We show that personalized LLMs trained using P-RLHF generate responses that are more closely aligned with individual user preferences.
  arXiv  Detail & Related papers  (2024-02-06T04:18:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.