ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response

• URL: http://arxiv.org/abs/2602.03430v2
• Date: Wed, 04 Feb 2026 03:41:04 GMT
• Title: ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response
• Abstract summary: ProAct-75 is a benchmark designed to train and evaluate proactive agents across diverse domains.<n>Our dataset features 91,581 step-level annotations enriched with explicit task graphs.<n>We propose ProAct-Helper, a reference baseline powered by a Multimodal Large Language Model (MLLM)
• Score: 20.913342340957904
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While passive agents merely follow instructions, proactive agents align with higher-level objectives, such as assistance and safety by continuously monitoring the environment to determine when and how to act. However, developing proactive agents is hindered by the lack of specialized resources. To address this, we introduce ProAct-75, a benchmark designed to train and evaluate proactive agents across diverse domains, including assistance, maintenance, and safety monitoring. Spanning 75 tasks, our dataset features 91,581 step-level annotations enriched with explicit task graphs. These graphs encode step dependencies and parallel execution possibilities, providing the structural grounding necessary for complex decision-making. Building on this benchmark, we propose ProAct-Helper, a reference baseline powered by a Multimodal Large Language Model (MLLM) that grounds decision-making in state detection, and leveraging task graphs to enable entropy-driven heuristic search for action selection, allowing agents to execute parallel threads independently rather than mirroring the human's next step. Extensive experiments demonstrate that ProAct-Helper outperforms strong closed-source models, improving trigger detection mF1 by 6.21%, saving 0.25 more steps in online one-step decision, and increasing the rate of parallel actions by 15.58%.

Related papers

• From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents [90.3279207163486]
  EvoSOP is a framework that empowers agents to extract SOPs from execution trajectories and iteratively optimize the toolset.<n>We show that EvoSOP significantly boosts task success rates while substantially reducing the number of interaction rounds.
  arXiv  Detail & Related papers  (2026-07-08T12:09:49Z)
• Anticipatory Planning for Multimodal AI Agents [77.62643381558613]
  We introduce TraceR1, a two-stage reinforcement learning framework that explicitly trains anticipatory reasoning.<n>TraceR1 is evaluated across seven benchmarks, covering online computer-use, offline computer-use benchmarks, and multimodal tool-use reasoning tasks.<n>Results show that anticipatory trajectory reasoning is a key principle for building multimodal agents that can reason, plan, and act effectively in complex real-world environments.
  arXiv  Detail & Related papers  (2026-03-17T16:55:11Z)
• ProactiveMobile: A Comprehensive Benchmark for Boosting Proactive Intelligence on Mobile Devices [17.39388308538324]
  This paper introduces ProactiveMobile, a benchmark for proactive mobile agent development.<n>It formalizes the proactive task as inferring latent user intent across four dimensions of on-device contextual signals.<n>The benchmark achieves a success rate of 19.15%, outperforming o1 (15.71%) and GPT-5 (7.39%) in experiments.
  arXiv  Detail & Related papers  (2026-02-25T12:32:37Z)
• Self-Abstraction from Grounded Experience for Plan-Guided Policy Refinement [61.35824395228412]
  Large language model (LLM) based agents are increasingly used to tackle software engineering tasks.<n>We propose Self-Abstraction from Grounded Experience (SAGE), a framework that enables agents to learn from their own task executions.
  arXiv  Detail & Related papers  (2025-11-08T08:49:38Z)
• OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks [52.87238755666243]
  We present OmniEAR, a framework for evaluating how language models reason about physical interactions, tool usage, and multi-agent coordination in embodied tasks.<n>We model continuous physical properties and complex spatial relationships across 1,500 scenarios spanning household and industrial domains.<n>Our systematic evaluation reveals severe performance degradation when models must reason from constraints.
  arXiv  Detail & Related papers  (2025-08-07T17:54:15Z)
• Pre-Act: Multi-Step Planning and Reasoning Improves Acting in LLM Agents [40.73340280747757]
  The ReAct capability in large language models (LLMs) has become the foundation of modern agentic systems.<n>We introduce Pre-Act, a novel approach that enhances the agent's performance by creating a multi-step execution plan.<n>Our approach is applicable to both conversational and non-conversational agents.
  arXiv  Detail & Related papers  (2025-05-15T05:17:47Z)
• Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance [95.03771007780976]
  We tackle the challenge of developing proactive agents capable of anticipating and initiating tasks without explicit human instructions.<n>First, we collect real-world human activities to generate proactive task predictions.<n>These predictions are labeled by human annotators as either accepted or rejected.<n>The labeled data is used to train a reward model that simulates human judgment.
  arXiv  Detail & Related papers  (2024-10-16T08:24:09Z)
• Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement [50.481380478458945]
  Iterative step-level Process Refinement (IPR) framework provides detailed step-by-step guidance to enhance agent training. Our experiments on three complex agent tasks demonstrate that our framework outperforms a variety of strong baselines.
  arXiv  Detail & Related papers  (2024-06-17T03:29:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.