Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answering

• URL: http://arxiv.org/abs/2605.12645v1
• Date: Tue, 12 May 2026 18:38:05 GMT
• Title: Training LLMs with Reinforcement Learning for Intent-Aware Personalized Question Answering
• Abstract summary: Intent-Aware Personalization (IAP) trains models to infer implicit user intent directly from a single-turn question.<n>IAP incorporates user intent into thinking steps through a tag-based schema for generating personalized, intent-grounded answers.<n>IAP consistently outperforms all baselines in experiments on the LaMP-QA benchmark.
• Score: 6.156855333798949
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effective personalized question answering (PQA) in language models requires grounding responses in the user's underlying intent, where intent refers to the implicit ``why'' behind a query beyond its explicit wording. However, existing approaches to intent-aware personalization rely on multi-turn conversational context or rich user profiles, and do not explicitly model user intent during the reasoning process. This limits their effectiveness in single-turn settings, where the user's latent goal must be inferred from minimal input and integrated into the thinking and reasoning process. To bridge this gap, we propose IAP (Intent-Aware Personalization), a reinforcement learning framework that trains models to infer implicit user intent directly from a single-turn question and incorporate it into thinking steps through a tag-based schema for generating personalized, intent-grounded answers. By optimizing intent-aware answer trajectories under a personalized reward function, IAP reinforces generation paths that make implicit user intent explicit and produce responses that better align with the user's underlying goal. Through experiments on the LaMP-QA benchmark across six models, IAP consistently outperforms all baselines, achieving an average macro-score gain of around 7.5\% over the strongest competitor, demonstrating that modeling implicit user intent within the training objective is a promising direction for PQA.

Related papers

• Intent Speaks Louder: Controllable User Simulation Beyond Response Imitation [25.422690765682788]
  controllable user simulation should separate which local interaction intent the next user turn should realize from how that intent is expressed in language.<n>We introduce UserIDA, which exposes interaction intent as an explicit per-turn directive.<n>On LMSYS-USP, UserIDA achieves 86.6% intent accuracy, outperforming the strongest dedicated user-simulator baseline by 24.3 percentage points.
  arXiv  Detail & Related papers  (2026-08-10T10:48:40Z)
• Preference-Aware Rubric Learning for Personalized Evaluation [59.539429430690156]
  Existing evaluation methods fail to capture user-specific preferences embedded in long-term interaction histories.<n>We propose Personalized Evaluation as Learning, a paradigm that formulates personalized evaluation as a learning problem rather than a static judgment.<n>Experiments show that PARL consistently induces high-fidelity rubrics that reliably identify user-aligned responses and generalize across users.
  arXiv  Detail & Related papers  (2026-05-29T17:00:55Z)
• IntPro: A Proxy Agent for Context-Aware Intent Understanding via Retrieval-conditioned Inference [22.214499732123173]
  IntPro is a proxy agent that learns to adapt to individual users via retrieval-conditioned intent inference.<n>We show that IntPro achieves strong intent understanding performance with effective context-aware reasoning capabilities.
  arXiv  Detail & Related papers  (2026-02-10T07:20:58Z)
• DiscoverLLM: From Executing Intents to Discovering Them [30.142994019166796]
  We introduce DiscoverLLM, a framework that trains Large Language Models to help users form and discover intents.<n>Resulting models learn to collaborate with users by adaptively diverging (i.e., explore options) when intents are unclear.<n>In a user study with 75 human participants, DiscoverLLM improved conversation satisfaction and efficiency compared to baselines.
  arXiv  Detail & Related papers  (2026-02-03T11:51:46Z)
• Learning Steerable Clarification Policies with Collaborative Self-play [67.67872810596839]
  To handle ambiguous queries, AI assistants need a policy for managing their uncertainty.<n>We propose to train steerable policies for managing this uncertainty using self-play.<n>We show this leads to a steerable policy that changes its behavior predictably conditioned on the provided costs.
  arXiv  Detail & Related papers  (2025-12-03T18:49:54Z)
• IPQA: A Benchmark for Core Intent Identification in Personalized Question Answering [13.337602043970051]
  We introduce the concept of core intents: intents users prioritize when selecting answers to satisfy their information needs.<n>Since users do not explicitly state their intents, we derive core intents from observable behavior patterns in answer selection.<n>We construct a dataset with various domains through systematic filtering, LLM-based annotation, and rigorous quality control.
  arXiv  Detail & Related papers  (2025-10-27T17:12:49Z)
• Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialogues [54.81155589931697]
  Collaborative Instance object Navigation (CoIN) is a new task setting where the agent actively resolve uncertainties about the target instance.<n>We propose a novel training-free method, Agent-user Interaction with UncerTainty Awareness (AIUTA)<n>First, upon object detection, a Self-Questioner model initiates a self-dialogue within the agent to obtain a complete and accurate observation description.<n>An Interaction Trigger module determines whether to ask a question to the human, continue or halt navigation.
  arXiv  Detail & Related papers  (2024-12-02T08:16:38Z)
• Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning [99.38163119531745]
  We show that applying a discretizing bottleneck can improve performance in goal-conditioned RL setups. We experimentally prove the expected return on out-of-distribution goals, while still allowing for specifying goals with expressive structure.
  arXiv  Detail & Related papers  (2022-11-01T03:31:43Z)
• Zero-Shot Prompting for Implicit Intent Prediction and Recommendation with Commonsense Reasoning [28.441725610692714]
  This paper proposes a framework of multi-domain dialogue systems, which can automatically infer implicit intents based on user utterances. The proposed framework is demonstrated effective to realize implicit intents and recommend associated bots in a zero-shot manner.
  arXiv  Detail & Related papers  (2022-10-12T03:33:49Z)
• Generative multitask learning mitigates target-causing confounding [61.21582323566118]
  We propose a simple and scalable approach to causal representation learning for multitask learning. The improvement comes from mitigating unobserved confounders that cause the targets, but not the input. Our results on the Attributes of People and Taskonomy datasets reflect the conceptual improvement in robustness to prior probability shift.
  arXiv  Detail & Related papers  (2022-02-08T20:42:14Z)
• Intent Contrastive Learning for Sequential Recommendation [86.54439927038968]
  We introduce a latent variable to represent users' intents and learn the distribution function of the latent variable via clustering. We propose to leverage the learned intents into SR models via contrastive SSL, which maximizes the agreement between a view of sequence and its corresponding intent. Experiments conducted on four real-world datasets demonstrate the superiority of the proposed learning paradigm.
  arXiv  Detail & Related papers  (2022-02-05T09:24:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.