UserHarness: Harnessing User Minds for Stronger Agent Theory-of-Mind

• URL: http://arxiv.org/abs/2605.27721v1
• Date: Tue, 26 May 2026 21:45:45 GMT
• Title: UserHarness: Harnessing User Minds for Stronger Agent Theory-of-Mind
• Abstract summary: UserHarness is a simple framework that reframes Theory-of-Mind (ToM) reasoning as explicit user-mind reconstruction.<n>It decomposes the user's mental state, its relation to the external environment, and the actions that follow from it, enabling agents to track what the user observes, believes, intends, and does.
• Score: 51.47914365631888
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding what a user believes and intends is central to building effective agent assistants. This ability is often evaluated through Theory-of-Mind (ToM) tasks, where success requires reasoning from the user's perspective. However, many existing approaches address ToM with complex pipelines that model behavior indirectly, without explicitly reconstructing the user's mental state. This misses the core structure of the problem: users act based on their beliefs, which are updated through observations of the environment; beliefs and intentions jointly determine actions, which in turn change the environment; and social reasoning often requires nested beliefs about what others believe or intend. We propose UserHarness, a simple framework that reframes ToM reasoning as explicit user-mind reconstruction. UserHarness decomposes the user's mental state, its relation to the external environment, and the actions that follow from it, enabling agents to track what the user observes, believes, intends, and does. Across five benchmarks, UserHarness reaches up to 95.94% macro accuracy, improving over existing inference methods by more than 15% relative and over the strongest prompt-only harness by about 20% relative. These results suggest that robust user understanding requires reasoning from the roots of the user's mind, positioning user harnessing as a promising foundation for more adaptive future assistants.

Related papers

• Inverse Theory of Mind Modeling for Content Recommendation: From Web Browsing to Dynamic Intelligent Interfaces [11.250965676125302]
  We propose an Inverse Theory of Mind (IToM) pipeline to infer the beliefs, preferences, and decision-making traits that explain behavior.<n>The pipeline reconstructs each user's decision context, including what was chosen and what alternatives were available.<n>We demonstrate cross-modal transferability with a persona-driven spatial banking application on VisionOS.
  arXiv  Detail & Related papers  (2026-08-11T18:58:28Z)
• Preference-Aware Rubric Learning for Personalized Evaluation [59.539429430690156]
  Existing evaluation methods fail to capture user-specific preferences embedded in long-term interaction histories.<n>We propose Personalized Evaluation as Learning, a paradigm that formulates personalized evaluation as a learning problem rather than a static judgment.<n>Experiments show that PARL consistently induces high-fidelity rubrics that reliably identify user-aligned responses and generalize across users.
  arXiv  Detail & Related papers  (2026-05-29T17:00:55Z)
• VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions [63.13827503828231]
  We introduce VitaBench 2.0, a benchmark for evaluating personalized and proactive agent behavior in long-term user interactions.<n>Results show that real-world personalization remains highly challenging even for state-of-the-art models.
  arXiv  Detail & Related papers  (2026-05-26T15:07:38Z)
• SAGER: Self-Evolving User Policy Skills for Recommendation Agent [18.821028533463824]
  Large language model (LLM) based recommendation agents personalize what they know through evolving per-user semantic memory.<n>We propose SAGER, the first recommendation agent framework in which each user is equipped with a dedicated policy skill.<n> Experiments on four public benchmarks demonstrate that SAGER achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-16T13:06:49Z)
• Beyond Words: Evaluating and Bridging Epistemic Divergence in User-Agent Interaction via Theory of Mind [8.740788873949471]
  Large Language Models (LLMs) have developed rapidly and are widely applied to both general-purpose and professional tasks.<n>They still struggle to comprehend and respond to the true user needs when intentions and instructions are imprecisely conveyed.
  arXiv  Detail & Related papers  (2026-02-14T16:01:59Z)
• UserBench: An Interactive Gym Environment for User-Centric Agents [110.77212949007958]
  Large Language Models (LLMs)-based agents have made impressive progress in reasoning and tool use, but their ability to proactively collaborate with users remains underexplored.<n>We introduce UserBench, a user-centric benchmark designed to evaluate agents in multi-turn, preference-driven interactions.
  arXiv  Detail & Related papers  (2025-07-29T17:34:12Z)
• Trust-Oriented Adaptive Guardrails for Large Language Models [9.719986610417441]
  Guardrails are designed to ensure that large language models (LLMs) align with human values by moderating harmful or toxic responses.<n>This paper addresses a critical issue: existing guardrails lack a well-founded methodology to accommodate the diverse needs of different user groups.<n>We introduce an adaptive guardrail mechanism, to dynamically moderate access to sensitive content based on user trust metrics.
  arXiv  Detail & Related papers  (2024-08-16T18:07:48Z)
• Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents [110.25679611755962]
  Current language model-driven agents often lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions. We introduce Intention-in-Interaction (IN3), a novel benchmark designed to inspect users' implicit intentions through explicit queries. We empirically train Mistral-Interact, a powerful model that proactively assesses task vagueness, inquires user intentions, and refines them into actionable goals.
  arXiv  Detail & Related papers  (2024-02-14T14:36:30Z)
• Diagnosis, Feedback, Adaptation: A Human-in-the-Loop Framework for Test-Time Policy Adaptation [20.266695694005943]
  Policies often fail due to distribution shift -- changes in the state and reward that occur when a policy is deployed in new environments. Data augmentation can increase robustness by making the model invariant to task-irrelevant changes in the agent's observation. We propose an interactive framework to leverage feedback directly from the user to identify personalized task-irrelevant concepts.
  arXiv  Detail & Related papers  (2023-07-12T17:55:08Z)
• Maximizing Information Gain in Partially Observable Environments via Prediction Reward [64.24528565312463]
  This paper tackles the challenge of using belief-based rewards for a deep RL agent. We derive the exact error between negative entropy and the expected prediction reward. This insight provides theoretical motivation for several fields using prediction rewards.
  arXiv  Detail & Related papers  (2020-05-11T08:13:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.