Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
- URL: http://arxiv.org/abs/2606.11042v3
- Date: Thu, 11 Jun 2026 16:59:56 GMT
- Title: Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
- Abstract summary: We introduce a benchmark for long-horizon GUI tasks centered on professional domains and specialized software environments.<n>We find that even the strongest models achieve only slightly above 30% success rates.<n>Our findings provide important insights into the limitations of current agent systems and suggest key directions for the next generation of GUI-agent research.
- Score: 91.11458140756208
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Recent years have witnessed the rapid evolution of AI agents toward handling increasingly complex, real-world tasks. However, existing benchmarks rarely evaluate whether agents can operate graphical user interfaces to complete long-horizon, high-value professional workflows across diverse domains. Current GUI benchmarks still predominantly focus on general-purpose software, relatively simple applications, and short-horizon tasks, leaving it largely unknown whether modern agents can follow user instructions to autonomously operate domain-specific professional software and accomplish economically valuable work in an end-to-end manner. To bridge this gap, we introduce Workflow-GYM, a benchmark for long-horizon GUI tasks centered on professional domains and specialized software environments. Through extensive experiments on state-of-the-art models, we find that even the strongest models achieve only slightly above 30% success rates, highlighting that professional long-horizon GUI workflows remain highly challenging for current GUI agents. Further analysis reveals that current agents struggle to maintain long-horizon workflow consistency, frequently exhibiting workflow stage omission, error propagation, objective drift, and insufficient understanding of professional software environments. Our findings provide important insights into the limitations of current agent systems and suggest key directions for the next generation of GUI-agent research.
Related papers
- StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows [63.57092302641323]
We introduce textbfStartupBench, an E2E agent benchmark grounded in market-validated AI startup products.<n>We study AI products with demonstrated adoption, together with their product and users, to identify real-world tasks for which AI has established practical demand.<n>Our results reveal that many market-validated agents remain beyond the reliable capabilities of current general-purpose agents.
arXiv Detail & Related papers (2026-08-18T14:01:32Z) - Running the Gauntlet: Re-evaluating the Capabilities of Agents Beyond Familiar Environments [67.2234316079859]
GauntletBench is a web-based benchmark for evaluating agent generalisation in challenging scenarios.<n>It focuses on three underexplored capabilities (temporal perception, graphical understanding, and 3D reasoning) across five less-covered professional applications.<n>Our empirical results reveal that frontier agentic systems remain far from achieving human-level performance.
arXiv Detail & Related papers (2026-06-12T12:32:24Z) - CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing [30.62336565964961]
Cutverse is a benchmark designed to evaluate autonomous GUI agents in realistic media post-production environments.<n>We curate expert demonstrations across 7 professional applications (e.g., Premiere Pro, Photoshop) covering complex, long-horizon tasks grounded in authentic editing.
arXiv Detail & Related papers (2026-05-19T07:35:22Z) - The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration [58.61149924275458]
Tool use enables large language models to access external information, invoke software systems, and act in digital environments beyond what can be solved from model parameters alone.<n>As agent systems evolve, the central problem has shifted from isolated invocation to multi-tool orchestration over long trajectories with intermediate state, execution feedback, changing environments, and practical constraints such as safety, cost, and verifiability.<n>We organize the literature around six core dimensions: inference-time planning and execution, training and trajectory construction, safety and control, efficiency under resource constraints, capability completeness in open environments, and benchmark design and evaluation.
arXiv Detail & Related papers (2026-03-24T07:05:05Z) - LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces [65.11019654023978]
LongCLI-Bench is a benchmark designed to evaluate agentic capabilities across long-horizon, realistic tasks.<n>We curated 20 high-quality, long-horizon tasks from over 1,000 computer science assignments and real-world tasks.<n>Experiments reveal that even state-of-the-art agents achieve pass rates below 20% in LongCLI-Bench.
arXiv Detail & Related papers (2026-02-15T23:12:57Z) - ProBench: Benchmarking GUI Agents with Accurate Process Information [15.519853892615272]
We introduce ProBench, a comprehensive benchmark with over 200 challenging GUI tasks covering widely-used scenarios.<n>We extend our dataset to include Process-related Task and design a specialized evaluation method.<n>Our evaluation of advanced GUI agents reveals significant limitations for real-world GUI scenarios.
arXiv Detail & Related papers (2025-11-12T09:49:31Z) - ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use [47.568491119335924]
We introduce ScreenSpot-Pro, a new benchmark designed to rigorously evaluate the grounding capabilities of MLLMs in high-resolution professional settings.<n>The benchmark comprises authentic high-resolution images from a variety of professional domains with expert annotations.<n>We propose ScreenSeekeR, a visual search method that utilizes the GUI knowledge of a strong planner to guide a cascaded search, achieving state-of-the-art performance with 48.1% without any additional training.
arXiv Detail & Related papers (2025-04-04T14:25:17Z) - WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks? [83.19032025950986]
We study the use of large language model-based agents for interacting with software via web browsers.
WorkArena is a benchmark of 33 tasks based on the widely-used ServiceNow platform.
BrowserGym is an environment for the design and evaluation of such agents.
arXiv Detail & Related papers (2024-03-12T14:58:45Z) - ASSISTGUI: Task-Oriented Desktop Graphical User Interface Automation [30.693616802332745]
This paper presents a novel benchmark, AssistGUI, to evaluate whether models are capable of manipulating the mouse and keyboard on the Windows platform in response to user-requested tasks.
We propose an advanced Actor-Critic framework, which incorporates a sophisticated GUI driven by an AI agent and adept at handling lengthy procedural tasks.
arXiv Detail & Related papers (2023-12-20T15:28:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.