Towards Better Agents for Multi-Turn User Interaction: The Next User Turn Is More Than Context

• URL: http://arxiv.org/abs/2608.17499v1
• Date: Tue, 18 Aug 2026 08:25:21 GMT
• Title: Towards Better Agents for Multi-Turn User Interaction: The Next User Turn Is More Than Context
• Abstract summary: Next-turn user reactions provide actionable local credit for improving multi-turn user-interacting agents.<n>textscFACA improves the nine-domain $$-family average across three independently trained runs.
• Score: 12.546026856710148
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: User-facing tool agents must coordinate dialogue and tool use as user goals unfold over multiple turns. Yet interactive reinforcement learning typically reduces each rollout to a terminal reward, assigning the same credit to effective elicitation, errors, and later repair. The next user turn is more than context: it also provides noisy, temporally local evidence about the preceding user-to-user segment. We introduce \textbf{F}eedback-\textbf{A}ware \textbf{C}redit \textbf{A}ssignment (\textsc{FACA}), which aligns each reaction with that segment, derives a locally normalized reaction advantage, and adds it to verified terminal outcome advantage without an extra critic or rollout. Against an outcome-only Interactive GRPO control matched in simulator, visible dialogue, initialization, rollout, and optimization, \textsc{FACA} improves the nine-domain $τ$-family average across three independently trained runs by 5.91 and 10.22 percentage points at 8B and 14B, respectively. Gains concentrate in Telecom; at 8B, randomizing reaction polarity removes the Telecom gain. The same ordering holds zero-shot on Pare-Bench and Co-Gym. These results demonstrate that next-turn user reactions provide actionable local credit for improving multi-turn user-interacting agents.

Related papers

• IAPO: Influence-Aware Policy Optimization for Credit Assignment in Multi-Turn Service Agents [10.577620258047665]
  This work advances the understanding of credit assignment in multi-turn user interactions.<n>It provides a principled approach to training service agents from sparse outcome feedback.
  arXiv  Detail & Related papers  (2026-08-25T14:14:36Z)
• ARC: Fair Relative Advantage Comparison in Open-Ended Real-World Interaction [4.6344273009233214]
  ARC is a novel paradigm for responsive, steerable, and execution-aware user-agent interaction.<n> ARC substantially strengthens the core $/2$ tool-use benchmarks.<n>inter reduces time-to-first-token from 4.91s to 1.27s relative to a think-style baseline.
  arXiv  Detail & Related papers  (2026-08-13T01:51:53Z)
• Intent Speaks Louder: Controllable User Simulation Beyond Response Imitation [25.422690765682788]
  controllable user simulation should separate which local interaction intent the next user turn should realize from how that intent is expressed in language.<n>We introduce UserIDA, which exposes interaction intent as an explicit per-turn directive.<n>On LMSYS-USP, UserIDA achieves 86.6% intent accuracy, outperforming the strongest dedicated user-simulator baseline by 24.3 percentage points.
  arXiv  Detail & Related papers  (2026-08-10T10:48:40Z)
• SWE-Together: Evaluating Coding Agents in Interactive User Sessions [19.069719245971786]
  We introduce SWE-Together, a multi-turn benchmark reconstructed from real user-agent coding sessions.<n>We curate 109 repository-level tasks from 11,260 recorded sessions, selecting sessions with recoverable repository states, clear user goals, and observable outcomes.<n>We build a reactive LLM-based user simulator that preserves the original users' intents and provides feedback when the coding agent's progress requires it.
  arXiv  Detail & Related papers  (2026-06-29T08:35:15Z)
• TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning [52.67756371231985]
  Reinforcement learning with verifiable rewards (RLVR) is a promising approach for enhancing reasoning and agentic behavior in large language models.<n>We introduce Tree Rollout Allocation for Contrastive Exploration (TRACE), a unified rollout allocation framework.<n>Technically, TRACE allocates rollout budget to both prompt roots and intermediate prefixes that are most likely to yield mixed terminal rewards.
  arXiv  Detail & Related papers  (2026-06-09T17:16:03Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation [72.01173512175531]
  KnowU-Bench is an online benchmark for personalized mobile agents.<n>It covers 42 general GUI tasks, 86 personalized tasks, and 64 proactive tasks.<n>Our experiments reveal a striking degradation: agents that excel at explicit task execution fall below 50% under vague instructions.
  arXiv  Detail & Related papers  (2026-04-09T16:50:50Z)
• PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments [72.02445514666428]
  We introduce PERMA, a benchmark designed to evaluate persona consistency over time beyond static preference recall.<n>PerMA consists of temporally ordered interaction events spanning multiple sessions and domains, with preference-related queries inserted over time.<n> Experiments demonstrate that by linking related interactions, advanced memory systems can extract more precise preferences and reduce token consumption.
  arXiv  Detail & Related papers  (2026-03-24T14:04:11Z)
• Listening to the Echo: User-Reaction Aware Policy Optimization via Scalar-Verbal Hybrid Reinforcement Learning [42.070531160001785]
  We propose Reaction Aware Policy optimization (RAPO) for emotional support systems.<n>RAPO treats dialogue as a reaction-driven process and utilizes simulated user responses to generate dense natural-language feedback.<n>RAPO significantly outperforms strong reinforcement learning baselines in driving positive interaction outcomes.
  arXiv  Detail & Related papers  (2026-03-16T15:39:54Z)
• OpenClaw-RL: Train Any Agent Simply by Talking [54.06773485601523]
  Next-state signals are universal, and policy can learn from all of them simultaneously.<n>Personal conversations, terminal executions, GUI interactions, SWE tasks, and tool-call traces are not separate training problems.<n>OpenClaw-RL enables an agent to improve simply by being used, recovering conversational signals from user re-queries, corrections, and explicit feedback.
  arXiv  Detail & Related papers  (2026-03-10T18:59:01Z)
• On the Mechanisms of Collaborative Learning in VAE Recommenders [12.77626224639472]
  Variational Autoencoders (VAEs) are a powerful alternative to matrix factorization for recommendation.<n>We analyze how collaboration arises in VAE-based collaborative filtering (CF) and show it is governed by latent proximity.<n>Our analyses are validated on the Netflix, MovieLens-20M, and Million Song datasets.
  arXiv  Detail & Related papers  (2025-11-10T07:13:58Z)
• UserBench: An Interactive Gym Environment for User-Centric Agents [110.77212949007958]
  Large Language Models (LLMs)-based agents have made impressive progress in reasoning and tool use, but their ability to proactively collaborate with users remains underexplored.<n>We introduce UserBench, a user-centric benchmark designed to evaluate agents in multi-turn, preference-driven interactions.
  arXiv  Detail & Related papers  (2025-07-29T17:34:12Z)
• Search-Based Interaction For Conversation Recommendation via Generative Reward Model Based Simulated User [117.82681846559909]
  Conversational recommendation systems (CRSs) use multi-turn interaction to capture user preferences and provide personalized recommendations.<n>We propose a generative reward model based simulated user, named GRSU, for automatic interaction with CRSs.
  arXiv  Detail & Related papers  (2025-04-29T06:37:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.