ProEvent: An Event-centric Benchmark for Proactive Agents

• URL: http://arxiv.org/abs/2607.17701v1
• Date: Mon, 20 Jul 2026 08:48:36 GMT
• Title: ProEvent: An Event-centric Benchmark for Proactive Agents
• Abstract summary: ProEvent is the first event-centric benchmark designed to assess an agent's ability to proactively maintain a user's timetable.<n>ProEvent provides synthesized yet realistic chats that consider the dynamic interaction among users, concurrent chat threads, and noise in the real world.
• Score: 41.03981463319731
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Proactive agents are expected to anticipate user needs and provide autonomous assistance by perceiving environmental context without explicit instructions. A fundamental capability of such agents is to identify and track users' upcoming events, enabling continuous and event-specific assistance. For example, by recording the time and location of a planned hike, an agent can deliver weather reminders in advance or provide navigation support before departure. However, existing works on proactive agents largely overlook event-centric assistance, and the open-ended nature of proactive assistance poses challenges for reliable evaluation. To bridge these gaps, we introduce ProEvent, the first event-centric benchmark designed to assess an agent's ability to proactively maintain a user's timetable based on ongoing instant messaging chats. ProEvent provides synthesized yet realistic chats that consider the dynamic interaction among users, concurrent chat threads, and noise in the real world, and evaluates proactive agents on response timing, single-step response correctness, and multi-step response correctness. Experiments on eight LLMs and pipelines reveal that current agents frequently overact and struggle with event cancellation. Notably, even GPT-5.1 only reacts correctly in 26.7% of scenarios. Further qualitative analysis reveals fundamental limitations of current LLMs as proactive agents, particularly in detecting implicit events and reasoning from the user's first-person perspective.

Related papers

• From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents [56.187590188698955]
  Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful.<n>Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles.<n>We introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories.<n>We propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment.
  arXiv  Detail & Related papers  (2026-08-03T12:52:04Z)
• Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile Agents [17.725926482772227]
  We propose the textbfPre-Reasoning Perception Framework (PRPF), a two-stage framework built on perceiving before reasoning.<n>PRPF introduces a lightweight Multimodal Proactive Perceptor (MPP) for intervention gating and context compression, and activates the Proactive Agent Reasoner (PAR) only when intervention is warranted.<n> Experiments on the ProactiveMobile benchmark show that PRPF substantially reduces false trigger rates (FTR) while improving success rates (SR) and inference efficiency over the ProactiveMobile baseline.
  arXiv  Detail & Related papers  (2026-06-02T06:54:02Z)
• Momento: Evaluating Persistent Memory and Reasoning with Multi-Session Agentic Conversations [14.480725554900777]
  We introduce Momento, a benchmark for persistent agentic task completion in multi-session service environments.<n> Experimental results reveal that current agents fail primarily through misestimation of user state.
  arXiv  Detail & Related papers  (2026-05-30T18:08:51Z)
• Anticipate and Learn: Unleashing Idle-Time Compute in Proactive Agents [51.46570338144688]
  ProAct is a proactive agent architecture that leverages idle-time compute to anticipate and fulfill likely upcoming user needs.<n>By analyzing evolving dialogue history together with persistent memory, ProAct predicts upcoming needs and iteratively acquires information.<n>MemBench evaluations confirm that ProAct achieves state-of-the-art reflective accuracy, underscoring its sustained and robust performance.
  arXiv  Detail & Related papers  (2026-05-25T15:47:21Z)
• Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization [61.641777037967366]
  Proactive large language model (LLM) agents aim to actively plan, query, and interact over multiple turns.<n>Agentic reinforcement learning (RL) has emerged as a promising solution for training such agents in multi-turn settings.<n>We propose BAO, an agentic RL framework that combines behavior enhancement to enrich proactive reasoning and information-gathering capabilities.
  arXiv  Detail & Related papers  (2026-02-11T20:40:43Z)
• ProAgent: Harnessing On-Demand Sensory Contexts for Proactive LLM Agent Systems [7.591337469415894]
  ProAgent is an end-to-end proactive agent system that harnesses massive sensory contexts and LLM reasoning to deliver proactive assistance.<n>We implement ProAgent on Augmented Reality (AR) glasses with an edge server and extensively evaluate it on a real-world testbed, a public dataset, and through a user study.<n>Results show that ProAgent achieves up to 33.4% higher proactive prediction accuracy, 16.8% higher tool-calling F1 score, and notable improvements in user satisfaction.
  arXiv  Detail & Related papers  (2025-12-07T08:21:07Z)
• Foundation Models for Logistics: Toward Certifiable, Conversational Planning Interfaces [59.80143393787701]
  Large language models (LLMs) can handle uncertainty and promise to accelerate replanning while lowering the barrier to entry.<n>We introduce a neurosymbolic framework that pairs the accessibility of natural-language dialogue with verifiable guarantees on goal interpretation.<n>A lightweight model, fine-tuned on just 100 uncertainty-filtered examples, surpasses the zero-shot performance of GPT-4.1 while cutting inference latency by nearly 50%.
  arXiv  Detail & Related papers  (2025-07-15T14:24:01Z)
• ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptions [8.300084456561171]
  We introduce ContextAgent, the first context-aware proactive agent.<n> ContextAgent extracts multi-dimensional contexts from sensory perceptions on wearables to understand user intentions.<n>When proactive assistance is needed, ContextAgent automatically calls the necessary tools to assist users unobtrusively.
  arXiv  Detail & Related papers  (2025-05-20T17:55:25Z)
• Interacting with Non-Cooperative User: A New Paradigm for Proactive Dialogue Policy [83.61404191470126]
  We propose a new solution named I-Pro that can learn Proactive policy in the Interactive setting. Specifically, we learn the trade-off via a learned goal weight, which consists of four factors. The experimental results demonstrate I-Pro significantly outperforms baselines in terms of effectiveness and interpretability.
  arXiv  Detail & Related papers  (2022-04-07T14:11:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.