CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing

• URL: http://arxiv.org/abs/2605.19484v1
• Date: Tue, 19 May 2026 07:35:22 GMT
• Title: CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing
• Abstract summary: Cutverse is a benchmark designed to evaluate autonomous GUI agents in realistic media post-production environments.<n>We curate expert demonstrations across 7 professional applications (e.g., Premiere Pro, Photoshop) covering complex, long-horizon tasks grounded in authentic editing.
• Score: 30.62336565964961
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While GUI agents have made significant progress in web navigation and basic operating system tasks, their capabilities in professional creative workflows remain largely underexplored. To bridge this gap, we introduce Cutverse, a benchmark designed to systematically evaluate autonomous GUI agents in realistic media post-production environments. We curate expert demonstrations across 7 professional applications (e.g., Premiere Pro, Photoshop), covering 186 complex, long-horizon tasks grounded in authentic editing workflows, involving dense multimodal interfaces and tightly coupled interaction sequences. To support scalable evaluation, we develop a lightweight parser that transforms raw screen recordings and low-level interaction logs into structured, compositional GUI action trajectories with precise grounding. Extensive evaluations reveal that existing agents achieve only 36.0\% task success on realistic media editing tasks, underscoring the challenges posed by complex, long-horizon media post-production workflows in our benchmark.While current models demonstrate promising spatial grounding, multimodal alignment, and coordinated action execution, they remain limited in long-horizon reliability and domain-specific planning.

Related papers

• JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition [51.15720718264743]
  We introduce JarvisGUI, a dynamic benchmark that evaluates GUI agents on cross-device requiring coordinated interaction across platforms.<n>JarvisGUI formulates GUI tasks as input-output transformations under a lightweight type system, which allows us to automatically compose multi-step, cross-device transformations.<n>We show that state-of-the-art open-source GUI agents struggle with the state-transfer awareness, cross-platform contextual reasoning, and long-horizon dependency management required for real-world usage.
  arXiv  Detail & Related papers  (2026-09-09T16:59:59Z)
• FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Production [55.71860113185132]
  We introduce FRAMEWORKERS, a task-centric and workspace-grounded multi-agent framework for open-ended video production.<n>A central Director formulates video creation as dynamic task management, continuously editing a Task Stack to determine which subtask to execute next and which sub-agent to invoke.<n>An Assistant serves as the execution layer, grounding each selected task in a shared Workspace, retrieving the required assets and context, invoking the assigned sub-agent, and persisting the resulting artifacts.<n>Experiments show that FRAMEWORKERS outperforms strong planners in routing accuracy, recovers reliably from runtime failures, generalizes to unseen sub
  arXiv  Detail & Related papers  (2026-08-30T14:28:50Z)
• Running the Gauntlet: Re-evaluating the Capabilities of Agents Beyond Familiar Environments [67.2234316079859]
  GauntletBench is a web-based benchmark for evaluating agent generalisation in challenging scenarios.<n>It focuses on three underexplored capabilities (temporal perception, graphical understanding, and 3D reasoning) across five less-covered professional applications.<n>Our empirical results reveal that frontier agentic systems remain far from achieving human-level performance.
  arXiv  Detail & Related papers  (2026-06-12T12:32:24Z)
• Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields [91.11458140756208]
  We introduce a benchmark for long-horizon GUI tasks centered on professional domains and specialized software environments.<n>We find that even the strongest models achieve only slightly above 30% success rates.<n>Our findings provide important insights into the limitations of current agent systems and suggest key directions for the next generation of GUI-agent research.
  arXiv  Detail & Related papers  (2026-06-09T16:10:16Z)
• GTA: Generating Long-Horizon Tasks for Web Agents at Scale [82.43869456830664]
  We introduce a scalable framework, GTA, that integrates crawling, retrieval-based seeding, in-context generation, and automated quality control.<n>We instantiate the pipeline on over 50 websites covering e-commerce, government, forums, and news, with multilingual and multi-hop coverage.<n>Our contributions are three-fold: (i) formalizing multi-hop web-agent task generation, (ii) proposing an efficient and validated pipeline for automatic data creation, and (iii) releasing a dynamic benchmark with reproducible evaluation.
  arXiv  Detail & Related papers  (2026-05-28T01:05:50Z)
• SaaS-Bench: Can Computer-Use Agents Leverage Real-World SaaS to Solve Professional Workflows? [29.5539656241093]
  Computer-Using Agents (CUAs) are rapidly extending large language models (LLMs) beyond text-based reasoning toward action execution in more complex environments.<n>Existing benchmarks often rely on simplified settings, isolated tasks, or short-horizon interactions.<n>We introduce a benchmark built on 23 deployable systems across six professional domains, containing 106 tasks grounded in realistic work scenarios.<n>Experiments show that representative LLM-based agents struggle on-Bench, with even the strongest model completing fewer than 4% of tasks end-to-end.
  arXiv  Detail & Related papers  (2026-05-15T09:35:15Z)
• CarePilot: A Multi-Agent Framework for Long-Horizon Computer Task Automation in Healthcare [37.42599407869901]
  Multimodal agentic pipelines are transforming human-computer interaction by enabling efficient and accessible automation of complex, real-world tasks.<n>Recent efforts have focused on short-horizon or general-purpose applications, leaving long-horizon automation for domain-specific systems, particularly in healthcare, largely unexplored.<n>We introduce CarePilot, a multi-agent framework based on the actor-critic paradigm.<n>Our experiments show that CarePilot achieves state-of-the-art performance, outperforming strong closed-source and open-source multimodal baselines by approximately 15.26% and 3.38%, respectively.
  arXiv  Detail & Related papers  (2026-03-25T10:25:48Z)
• VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chaining [49.78591189918702]
  We introduce VisualToolChain-Bench (VTC-Bench) to evaluate tool-use proficiency in Multimodal Large Language Models (MLLMs)<n>To align with realistic computer vision pipelines, our framework features 32 diverse OpenCV-based visual operations.<n>For precise evaluation, we provide 680 curated problems structured across a nine-category cognitive hierarchy.
  arXiv  Detail & Related papers  (2026-03-16T09:31:44Z)
• AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios [32.58358574768901]
  Real-world multimodal agents solve multi-step grounded in visual evidence.<n>Existing benchmarks mainly evaluate single-turn visual reasoning or specific tool skills.<n>We introduce AgentVista, a benchmark for generalist multimodal agents.
  arXiv  Detail & Related papers  (2026-02-26T16:30:46Z)
• ANCHOR: Branch-Point Data Generation for GUI Agents [52.22377425487]
  End-to-end GUI agents for real desktop environments require large amounts of high-quality interaction data.<n>We present a trajectory expansion framework Anchor that bootstraps scalable desktop supervision from a small set of verified seed demonstrations.<n>Experiments on standard desktop benchmarks, OSWorld and WindowsAgentArena, show that models fine-tuned on our expanded corpus achieve consistent improvements.
  arXiv  Detail & Related papers  (2026-02-06T19:55:26Z)
• GUI-360$^\circ$: A Comprehensive Dataset and Benchmark for Computer-Using Agents [59.107657859025586]
  GUI-360$circ$ is a large-scale, comprehensive dataset and benchmark suite designed to advance computer-using agents (CUAs)<n>The released corpus contains over 1.2M executed action steps across thousands of trajectories in popular Windows office applications.<n>The dataset supports three canonical tasks, GUI grounding, screen parsing, and action prediction, and a hybrid GUI+API action space.
  arXiv  Detail & Related papers  (2025-11-06T12:19:02Z)
• Hollywood Town: Long-Video Generation via Cross-Modal Multi-Agent Orchestration [73.65102758687289]
  This study introduces three innovations to improve multi-agent collaboration.<n>First, we propose OmniAgent, a hierarchical, graph-based multi-agent framework for long video generation.<n>Second, inspired by context engineering, we propose hypergraph nodes that enable temporary group discussions.
  arXiv  Detail & Related papers  (2025-10-25T20:34:18Z)
• AppAgent v2: Advanced Agent for Flexible Mobile Interactions [57.98933460388985]
  This work introduces a novel LLM-based multimodal agent framework for mobile devices.<n>Our agent constructs a flexible action space that enhances adaptability across various applications.<n>Our results demonstrate the framework's superior performance, confirming its effectiveness in real-world scenarios.
  arXiv  Detail & Related papers  (2024-08-05T06:31:39Z)
• ASSISTGUI: Task-Oriented Desktop Graphical User Interface Automation [30.693616802332745]
  This paper presents a novel benchmark, AssistGUI, to evaluate whether models are capable of manipulating the mouse and keyboard on the Windows platform in response to user-requested tasks. We propose an advanced Actor-Critic framework, which incorporates a sophisticated GUI driven by an AI agent and adept at handling lengthy procedural tasks.
  arXiv  Detail & Related papers  (2023-12-20T15:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.