WARC-Bench: Web Archive Based Benchmark for GUI Subtask Executions

• URL: http://arxiv.org/abs/2510.09872v1
• Date: Fri, 10 Oct 2025 21:20:51 GMT
• Title: WARC-Bench: Web Archive Based Benchmark for GUI Subtask Executions
• Abstract summary: Training web agents to navigate complex, real-world websites requires them to master $itsubtasks$.<n>We introduce WARC-Bench, a novel web navigation benchmark featuring 438 tasks designed to evaluate multimodal AI agents on subtasks.<n>We show that WARC-Bench is challenging for leading computer-use models, with the highest observed success rate being 64.8%.
• Score: 14.49822563708463
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Training web agents to navigate complex, real-world websites requires them to master $\textit{subtasks}$ - short-horizon interactions on multiple UI components (e.g., choosing the correct date in a date picker, or scrolling in a container to extract information). We introduce WARC-Bench (Web Archive Benchmark), a novel web navigation benchmark featuring 438 tasks designed to evaluate multimodal AI agents on subtasks. WARC-Bench enables sandboxed interactions with dynamic and realistic webpages using Web ARChive files. We show that WARC-Bench is challenging for leading computer-use models, with the highest observed success rate being 64.8%. To improve open source models on subtask, we explore two common training techniques: supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR). Experiments show that SFT models obtain a 48.8% success rate on the benchmark. Training with RLVR over SFT checkpoints, even in data-scarce settings, improves the score to 52.8% on WARC-Bench, outperforming many frontier models. Our analysis concludes that mastering these subtasks is essential for robust web planning and navigation, and is a capability not extensively evaluated by existing benchmarks.

Related papers

• Wuying-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agents [34.59625581903402]
  Wuying-Browser-Agent is a unified framework that addresses execution, supervision, optimization, and evaluation.<n>A structured browser harness provides stable execution primitives and decision-oriented context management.<n> BrowserBench is a benchmark of 350 tasks averaging 37.9 steps.
  arXiv  Detail & Related papers  (2026-08-18T03:23:31Z)
• UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations [68.29650478987561]
  UI-Mate is a foundation GUI agent that integrates an environment-grounded training stack with in-context demonstration learning.<n>It sets a new open-weight state of the art on general computer-use benchmarks, scoring 77.0% on OSWorld and 66.2% on WindowsAgentArena.
  arXiv  Detail & Related papers  (2026-08-16T20:59:59Z)
• SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL [38.05711358650287]
  Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital through visual perception and GUI execution.<n>Online reinforcement learning with verifiable rewards (RLVR) has emerged as a key direction for scaling their capabilities.<n>We introduce ScaleCUA, a unified framework that scales online RL for CUAs via verifiable task synthesis and efficient training.
  arXiv  Detail & Related papers  (2026-07-13T07:32:35Z)
• FastContext: Training Efficient Repository Explorer for Coding Agents [29.359764163872473]
  Large Language Model (LLM) coding agents have achieved strong results on software engineering tasks.<n> repository exploration remains a major bottleneck: locating relevant code consumes substantial token budget and pollutes the agent's context with irrelevant snippets.<n>We present FastContext, a dedicated exploration subagent that separates repository exploration from solving.
  arXiv  Detail & Related papers  (2026-06-12T03:23:59Z)
• MTA-Agent: An Open Recipe for Multimodal Deep Search Agents [51.180338423927985]
  Multi-hop large language models (MLLMs) have demonstrated strong capabilities in visual understanding, yet they remain limited in complex, multi-step reasoning.<n>We propose a Multi-hop Tool-Augmented Agent for Evidence-based QA Synthesis (MTA-Agent)<n>MTA-Agent automatically selects tools and their parameters to retrieve and validate evidence from both visual and textual sources.
  arXiv  Detail & Related papers  (2026-04-07T19:01:45Z)
• VisBrowse-Bench: Benchmarking Visual-Native Search for Multimodal Browsing Agents [27.574370658412864]
  We introduce a new benchmark for visual-native search, VisBrowse-Bench.<n>It contains 169 VQA instances covering multiple domains.<n>It evaluates the models' visual reasoning capabilities during the search process.
  arXiv  Detail & Related papers  (2026-03-17T09:24:13Z)
• ARC: Compiling Hundreds of Requirement Scenarios into A Runnable Web System [10.49539144799249]
  Agentic Requirement Compilation (ARC) is a technique that moves beyond simple code generation to requirement compilation.<n>ARC generates not only source code but also modular designs for UI, API, and database layers.<n>A user study with 21 participants showed that novice users can successfully write DSL documents for complex systems.
  arXiv  Detail & Related papers  (2026-02-14T11:07:58Z)
• AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents [49.67355440164857]
  We introduce AIRS-Bench, a suite of 20 tasks sourced from state-of-the-art machine learning papers.<n>Airs-Bench tasks assess agentic capabilities over the full research lifecycle.<n>We open-source the AIRS-Bench task definitions and evaluation code to catalyze further development in autonomous scientific research.
  arXiv  Detail & Related papers  (2026-02-06T16:45:02Z)
• AUTO-Explorer: Automated Data Collection for GUI Agent [58.58097564914626]
  We propose an automated data collection method with minimal annotation costs, named Auto-Explorer.<n>It incorporates a simple yet effective exploration mechanism that autonomously parses and explores GUI environments.<n>Using the data gathered, we fine-tune a multimodal large language model (MLLM) and establish a GUI element grounding testing set.
  arXiv  Detail & Related papers  (2025-11-09T15:13:45Z)
• WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents [57.203515352080295]
  We introduce WebExplorer: a systematic data generation approach using model-based exploration and iterative, long-to-short query evolution.<n>Our model supports 128K context length and up to 100 tool calling turns, enabling long-horizon problem solving.<n>As an 8B-sized model, WebExplorer-8B is able to effectively search over an average of 16 turns after RL training.
  arXiv  Detail & Related papers  (2025-09-08T10:07:03Z)
• WebSight: A Vision-First Architecture for Robust Web Agents [0.0]
  WebSight is a vision-based web agent designed to interact with web environments purely through visual perception.<n>We introduce WebSight-7B, a fine-tuned vision-language model optimized for UI element interaction.<n>WebSight-7B achieves a top-1 accuracy of 58.84% on the Showdown Clicks benchmark, outperforming several larger generalist models.<n>WebSight and WebSight-7B establish a new standard for interpretable, robust, and efficient visual web navigation.
  arXiv  Detail & Related papers  (2025-08-23T11:02:59Z)
• DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning [73.68685269970844]
  We introduce WebPuzzle, a training and 275-sample test benchmark that evaluates information seeking on the live internet.<n>We develop DeepDiver, a reinforcement-learning framework that cultivates Search Intensity Scaling (SIS)-an emergent ability to escalate search frequency and depth.<n>We detail DeepDiver's curriculum from cold-start SFT to a well designed RL procedure, and show that its seeking policy generalized from closed-ended queries to open-ended generation such as long-form writing.
  arXiv  Detail & Related papers  (2025-05-30T08:15:39Z)
• VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search [42.560419395815146]
  Vision-Language Models have made significant progress on many perception-focused tasks.<n>However, their progress on reasoning-focused tasks remains limited due to the lack of high-quality and diverse training data.<n>We propose VisualWebInstruct to create a diverse and high-quality dataset spanning multiple disciplines.
  arXiv  Detail & Related papers  (2025-03-13T17:32:48Z)
• DSBench: How Far Are Data Science Agents from Becoming Data Science Experts? [58.330879414174476]
  We introduce DSBench, a benchmark designed to evaluate data science agents with realistic tasks.<n>This benchmark includes 466 data analysis tasks and 74 data modeling tasks, sourced from Eloquence and Kaggle competitions.<n>Our evaluation of state-of-the-art LLMs, LVLMs, and agents shows that they struggle with most tasks, with the best agent solving only 34.12% of data analysis tasks and achieving a 34.74% Relative Performance Gap (RPG)
  arXiv  Detail & Related papers  (2024-09-12T02:08:00Z)
• PLANRL: A Motion Planning and Imitation Learning Framework to Bootstrap Reinforcement Learning [13.564676246832544]
  We introduce PLANRL, a framework that chooses when the robot should use classical motion planning and when it should learn a policy. PLANRL switches between two modes of operation: reaching a waypoint using classical techniques when away from the objects and fine-grained manipulation control when about to interact with objects. We evaluate our approach across multiple challenging simulation environments and real-world tasks, demonstrating superior performance in terms of adaptability, efficiency, and generalization compared to existing methods.
  arXiv  Detail & Related papers  (2024-08-07T19:30:08Z)
• DiscoveryBench: Towards Data-Driven Discovery with Large Language Models [50.36636396660163]
  We present DiscoveryBench, the first comprehensive benchmark that formalizes the multi-step process of data-driven discovery. Our benchmark contains 264 tasks collected across 6 diverse domains, such as sociology and engineering. Our benchmark, thus, illustrates the challenges in autonomous data-driven discovery and serves as a valuable resource for the community to make progress.
  arXiv  Detail & Related papers  (2024-07-01T18:58:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.