From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents

• URL: http://arxiv.org/abs/2608.02171v1
• Date: Mon, 03 Aug 2026 12:52:04 GMT
• Title: From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents
• Abstract summary: Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful.<n>Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles.<n>We introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories.<n>We propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment.
• Score: 56.187590188698955
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful. Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles. Such designs fail to capture the complexity of evolving user preferences and neglect preference-conditioned task execution-a discrepancy we term as the knowledge-to-action gap. To address this challenge, we introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories that contain noise, implicit cues, and temporal inconsistencies. Unlike prior work, IBA-Bench evaluates whether an agent can execute tasks while satisfying implicit user constraints inferred from historical interactions. We further propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment. Experiment results on IBA-Bench show that effective personalization remains a significant challenge for state-of-the-art LLM agents, and the proposed IBA-Agent substantially improves behavioral alignment in complex scenarios across nine application domains.

Related papers

• Intent-Driven Situation Tracking for User-Centric Multi-Turn Agents [4.138547191663224]
  We propose Intent-Driven Situation States (IDSS), a training-free framework that maintains an explicit situation state alongside the dialogue.<n>IDSS parses tool returns into provenance-aware entities and attributes, tracks user intents, required variables, constraints, and execution status, and propagates new facts to task constraints to update action executability.
  arXiv  Detail & Related papers  (2026-08-16T14:14:07Z)
• ODYSSE: Episode-wise Policy Optimization for Personalized Agentic Reasoning [48.692251671993574]
  We present ODYSSE, a framework for personalized agentic reasoning.<n>ODYSSE addresses long action horizons and strong cross-step dependencies in personalized agentic reasoning.<n>We show that ODYSSE consistently outperforms both specialist and general-purpose LVLMs.
  arXiv  Detail & Related papers  (2026-07-28T07:25:04Z)
• APeB: Benchmarking Personalization Ability of Large Language Model Agents [17.77241512443464]
  LLM-powered agents struggle with personalization when users issue raw, underspecified queries.<n>Existing benchmarks rarely test this capability, as they often rely on user-refined queries or simplified histories.<n>We introduce personalized product search (PPS), a testbed for agentic personalization under raw queries and diverse histories.
  arXiv  Detail & Related papers  (2026-07-03T10:03:35Z)
• Momento: Evaluating Persistent Memory and Reasoning with Multi-Session Agentic Conversations [14.480725554900777]
  We introduce Momento, a benchmark for persistent agentic task completion in multi-session service environments.<n> Experimental results reveal that current agents fail primarily through misestimation of user state.
  arXiv  Detail & Related papers  (2026-05-30T18:08:51Z)
• VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions [63.13827503828231]
  We introduce VitaBench 2.0, a benchmark for evaluating personalized and proactive agent behavior in long-term user interactions.<n>Results show that real-world personalization remains highly challenging even for state-of-the-art models.
  arXiv  Detail & Related papers  (2026-05-26T15:07:38Z)
• FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environments [60.3427704389541]
  Large Language Models are being increasingly deployed as the decision-making core of autonomous agents.<n>Yet, in conversational benchmarks, these agents frequently fail due to the cascading effects of incorrect decision-making.<n>We present the Failure-Aware Meta-Agentic framework to tackle these challenges.
  arXiv  Detail & Related papers  (2026-04-28T02:21:53Z)
• The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
  Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
  arXiv  Detail & Related papers  (2026-01-21T15:22:21Z)
• PersonaAgent: When Large Language Model Agents Meet Personalization at Test Time [87.99027488664282]
  PersonaAgent is a framework designed to address versatile personalization tasks.<n>It integrates a personalized memory module and a personalized action module.<n>Test-time user-preference alignment strategy ensures real-time user preference alignment.
  arXiv  Detail & Related papers  (2025-06-06T17:29:49Z)
• AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems [112.76941157194544]
  We propose AgentCF for simulating user-item interactions in recommender systems through agent-based collaborative filtering. We creatively consider not only users but also items as agents, and develop a collaborative learning approach that optimize both kinds of agents together. Overall, the optimized agents exhibit diverse interaction behaviors within our framework, including user-item, user-user, item-item, and collective interactions.
  arXiv  Detail & Related papers  (2023-10-13T16:37:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.