The Tool Illusion: Rethinking Tool Use in Web Agents

• URL: http://arxiv.org/abs/2604.03465v1
• Date: Fri, 03 Apr 2026 21:18:26 GMT
• Title: The Tool Illusion: Rethinking Tool Use in Web Agents
• Abstract summary: We revisit tool use in web agents through an extensive and carefully controlled study.<n>Our findings both revise some prior conclusions and complement others with broader evidence.
• Score: 60.49303190288166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As web agents rapidly evolve, an increasing body of work has moved beyond conventional atomic browser interactions and explored tool use as a higher-level action paradigm. Although prior studies have shown the promise of tools, their conclusions are often drawn from limited experimental scales and sometimes non-comparable settings. As a result, several fundamental questions remain unclear: i) whether tools provide consistent gains for web agents, ii) what practical design principles characterize effective tools, and iii) what side effects tool use may introduce. To establish a stronger empirical foundation for future research, we revisit tool use in web agents through an extensive and carefully controlled study across diverse tool sources, backbone models, tool-use frameworks, and evaluation benchmarks. Our findings both revise some prior conclusions and complement others with broader evidence. We hope this study provides a more reliable empirical basis and inspires future research on tool-use web agents.

Related papers

• Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents [8.575257729172202]
  Visual agents employ external visual tools within visual chains of thought to incorporate fine-grained evidence.<n>In this paper, we move beyond simple visual search tasks to investigate more challenging tasks, including 3D spatial reasoning and medical visual question answering.<n>We identify a tool-use collapse phenomenon: models progressively stop using tools while still achieving higher task accuracy.<n>We find that vanilla training and tool-use encouragement both reduce rollout diversity, explaining why higher tool use does not yield stronger reasoning performance.
  arXiv  Detail & Related papers  (2026-05-25T13:06:59Z)
• ARMOR: An Agentic Framework for Reaction Feasibility Prediction via Adaptive Utility-aware Multi-tool Reasoning [11.65859277511507]
  ARMOR is an agentic framework that explicitly models tool-specific utilities, adaptively prioritizes tools, and resolves potential tool conflicts.<n>Experiments on a public dataset demonstrate that ARMOR consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-05-08T01:30:20Z)
• Agentic Tool Use in Large Language Models [33.63702247846896]
  This paper organizes the literature into three paradigms: prompting as plug-and-play, supervised tool learning and reward-driven tool policy learning.<n>It aims to address this fragmentation and provide a more structured evolutionary view of agentic tool use.
  arXiv  Detail & Related papers  (2026-04-01T12:46:51Z)
• Which Tool Response Should I Trust? Tool-Expertise-Aware Chest X-ray Agent with Multimodal Agentic Learning [11.117796080402044]
  This paper introduces a framework that enables an agent to interact with tools and empirically learn their practical trustworthiness.<n>As a concrete instantiation, we focus on chest X-ray analysis and present a tool-expertise-aware chest X-ray agent.<n>When tool outputs disagree, the agent experimentally accepts or rejects multimodal tool results, receives rewards, and learns which tool to trust for each query type.
  arXiv  Detail & Related papers  (2026-02-25T03:09:47Z)
• AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning [66.24374176797075]
  We introduce textbfAdaReasoner, a family of multimodal models that learn tool use as a general reasoning skill rather than as tool-specific or explicitly supervised behavior.<n>AdaReasoner is enabled by (i) a scalable data curation pipeline exposing models to long-horizon, multi-step tool interactions; (ii) Tool-GRPO, a reinforcement learning algorithm that prioritizes tool selection and sequencing based on end-task success; and (iii) an adaptive learning mechanism that dynamically regulates tool usage.
  arXiv  Detail & Related papers  (2026-01-26T16:04:43Z)
• Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning [63.071280297939005]
  We present Transductive Visual Programming (TVP), a novel framework that builds new tools from its own experience rather than speculation.<n>TVP achieves state-of-the-art performance, outperforming GPT-4o by 22% and the previous best visual programming system by 11%.<n>Our work establishes experience-driven transductive tool creation as a powerful paradigm for building self-evolving visual programming agents.
  arXiv  Detail & Related papers  (2025-12-24T04:30:21Z)
• TheMCPCompany: Creating General-purpose Agents with Task-specific Tools [12.249551019598442]
  We introduce TheMCPCompany, a benchmark for evaluating tool-calling agents on tasks that involve interacting with various real-world services.<n>We also provide manually annotated ground-truth tools for each task.<n>Overall, our work shows that the most advanced reasoning models are effective at discovering tools in simpler environments, but seriously struggle with navigating complex enterprise environments.
  arXiv  Detail & Related papers  (2025-10-22T06:42:01Z)
• PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature [11.804526152911386]
  We propose PaperArena, an evaluation benchmark for large language model (LLM) based agents to address real-world research questions.<n>Given a research question, agents should integrate diverse formats across multiple papers through reasoning and interacting with appropriate tools.<n> Experimental results reveal that even the most advanced LLM powering a well-established agent system achieves merely 38.78% average accuracy.
  arXiv  Detail & Related papers  (2025-10-13T02:10:39Z)
• Towards Completeness-Oriented Tool Retrieval for Large Language Models [60.733557487886635]
  Real-world systems often incorporate a wide array of tools, making it impractical to input all tools into Large Language Models. Existing tool retrieval methods primarily focus on semantic matching between user queries and tool descriptions. We propose a novel modelagnostic COllaborative Learning-based Tool Retrieval approach, COLT, which captures not only the semantic similarities between user queries and tool descriptions but also takes into account the collaborative information of tools.
  arXiv  Detail & Related papers  (2024-05-25T06:41:23Z)
• EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction [56.02100384015907]
  EasyTool is a framework transforming diverse and lengthy tool documentation into a unified and concise tool instruction. It can significantly reduce token consumption and improve the performance of tool utilization in real-world scenarios.
  arXiv  Detail & Related papers  (2024-01-11T15:45:11Z)
• Tool Learning with Foundation Models [158.8640687353623]
  With the advent of foundation models, AI systems have the potential to be equally adept in tool use as humans. Despite its immense potential, there is still a lack of a comprehensive understanding of key challenges, opportunities, and future endeavors in this field.
  arXiv  Detail & Related papers  (2023-04-17T15:16:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.