Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants

• URL: http://arxiv.org/abs/2604.00842v1
• Date: Wed, 01 Apr 2026 12:53:01 GMT
• Title: Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants
• Abstract summary: Pare is a framework for building and evaluating proactive agents in digital environments.<n>Pare-Bench is a benchmark of 143 diverse tasks spanning communication, productivity, scheduling, and lifestyle apps.
• Score: 85.1155076383488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Proactive agents that anticipate user needs and autonomously execute tasks hold great promise as digital assistants, yet the lack of realistic user simulation frameworks hinders their development. Existing approaches model apps as flat tool-calling APIs, failing to capture the stateful and sequential nature of user interaction in digital environments and making realistic user simulation infeasible. We introduce Proactive Agent Research Environment (Pare), a framework for building and evaluating proactive agents in digital environments. Pare models applications as finite state machines with stateful navigation and state-dependent action space for the user simulator, enabling active user simulation. Building on this foundation, we present Pare-Bench, a benchmark of 143 diverse tasks spanning communication, productivity, scheduling, and lifestyle apps, designed to test context observation, goal inference, intervention timing, and multi-app orchestration.

Related papers

• DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows [69.12339622053588]
  We introduce DuMateBench, a real-session benchmark reconstructed from anonymized and privacy-screened user sessions.<n>The resulting benchmark comprises 200 tasks spanning 8 broad scenarios and 17 fine-grained capability categories.<n>We execute these tasks in isolated Docker containers injected with three forms of real-world environmental complexity: Insufficient, Unstable, and Noisy.
  arXiv  Detail & Related papers  (2026-08-27T02:36:23Z)
• VISTA: A Versatile Interactive User Simulation Toolkit for Agent Evaluation [16.74971373243012]
  Existing evaluation methods often rely on static benchmarks, which fail to capture the dynamic, multi-step nature of agentic behavior.<n>We propose VISTA, a Versatile Interactive user Simulation Toolkit for Agent evaluation.<n>Our toolkit includes a suite of six metrics for measuring the realism, capability coverage, and interaction effectiveness of simulated interactions.
  arXiv  Detail & Related papers  (2026-06-09T16:39:32Z)
• VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions [63.13827503828231]
  We introduce VitaBench 2.0, a benchmark for evaluating personalized and proactive agent behavior in long-term user interactions.<n>Results show that real-world personalization remains highly challenging even for state-of-the-art models.
  arXiv  Detail & Related papers  (2026-05-26T15:07:38Z)
• Computer-Using World Model [58.59112582915026]
  We introduce the Computer-Using World Model (CUWM), a world model for desktop software that predicts the next user interface (UI) state.<n> CUWM first predicts a textual description of agent-relevant state changes, and then realizes these changes visually to synthesize the next screenshot.<n>We evaluate CUWM via test-time action search, where a frozen agent uses the world model to simulate and compare candidate actions before execution.
  arXiv  Detail & Related papers  (2026-02-19T13:48:29Z)
• Long-term Task-oriented Agent: Proactive Long-term Intent Maintenance in Dynamic Environments [8.937298475124484]
  Current large language model agents operate under a reactive paradigm, responding only to immediate user queries within short-term sessions.<n>We propose a novel interaction paradigm for proactive Task-oriented Agents capable of bridging the gap between relatively static user's needs and a dynamic environment.<n>We introduce a high-quality data synthesis pipeline to construct complex, multi-turn dialog data in a dynamic environment.
  arXiv  Detail & Related papers  (2026-01-14T11:15:40Z)
• TongSIM: A General Platform for Simulating Intelligent Machines [59.27575233453533]
  Embodied intelligence focuses on training agents within realistic simulated environments.<n>TongSIM is a high-fidelity, general-purpose platform for training and evaluating embodied agents.
  arXiv  Detail & Related papers  (2025-12-23T10:00:43Z)
• Non-Collaborative User Simulators for Tool Agents [12.294827535425414]
  We propose a novel user simulator architecture that simulates four categories of non-collaborative behaviors.<n>Our experiments on MultiWOZ and $tau$-bench reveal significant performance degradation in state-of-the-art tool agents when encountering non-collaborative users.
  arXiv  Detail & Related papers  (2025-09-27T05:06:17Z)
• Test Automation for Interactive Scenarios via Promptable Traffic Simulation [48.240394447516664]
  We introduce an automated method to generate realistic and safety-critical human behaviors for AV planner evaluation in interactive scenarios.<n>We parameterize complex human behaviors using low-dimensional goal positions, which are then fed into a promptable traffic simulator, ProSim.<n>To automate test generation, we introduce a prompt generation module that explores the goal domain and efficiently identifies safety-critical behaviors using Bayesian optimization.
  arXiv  Detail & Related papers  (2025-06-01T22:29:32Z)
• Towards a Formal Characterization of User Simulation Objectives in Conversational Information Access [15.54070473873364]
  User simulation is a promising approach for automatically training and evaluating conversational information access agents. We define the distinct objectives for user simulators: training aims to maximize behavioral similarity to real users, while evaluation focuses on the accurate prediction of real-world conversational agent performance.
  arXiv  Detail & Related papers  (2024-06-27T08:46:41Z)
• USimAgent: Large Language Models for Simulating Search Users [33.17004578463697]
  We introduce a Large Language Models-based user search behavior simulator, USimAgent. The simulator can simulate users' querying, clicking, and stopping behaviors during search. Empirical investigation on a real user behavior dataset shows that the simulator outperforms existing methods in query generation.
  arXiv  Detail & Related papers  (2024-03-14T07:40:54Z)
• User Behavior Simulation with Large Language Model based Agents [116.74368915420065]
  We propose an LLM-based agent framework and design a sandbox environment to simulate real user behaviors. Based on extensive experiments, we find that the simulated behaviors of our method are very close to the ones of real humans.
  arXiv  Detail & Related papers  (2023-06-05T02:58:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.