Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentation

• URL: http://arxiv.org/abs/2506.19998v1
• Date: Tue, 24 Jun 2025 20:30:44 GMT
• Title: Doc2Agent: Scalable Generation of Tool-Using Agents from API Documentation
• Abstract summary: Doc2Agent is a scalable pipeline to build tool agents that can call Python-based tools generated from API documentation.<n>We evaluate our approach on real-world APIs, WebArena APIs, and research APIs, producing validated tools.
• Score: 2.4117201298131232
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: REST APIs play important roles in enriching the action space of web agents, yet most API-based agents rely on curated and uniform toolsets that do not reflect the complexity of real-world APIs. Building tool-using agents for arbitrary domains remains a major challenge, as it requires reading unstructured API documentation, testing APIs and inferring correct parameters. We propose Doc2Agent, a scalable pipeline to build agents that can call Python-based tools generated from API documentation. Doc2Agent generates executable tools from API documentations and iteratively refines them using a code agent. We evaluate our approach on real-world APIs, WebArena APIs, and research APIs, producing validated tools. We achieved a 55\% relative performance improvement with 90\% lower cost compared to direct API calling on WebArena benchmark. A domain-specific agent built for glycomaterial science further demonstrates the pipeline's adaptability to complex, knowledge-rich tasks. Doc2Agent offers a generalizable solution for building tool agents from unstructured API documentation at scale.

Related papers

• AISE-Bench: A Full-Cycle Curated Benchmark for Information Seeking on Academic Knowledge Graphs [55.8213573429699]
  AISE-Bench is a real-world, full-cycle annotated benchmark for information seeking on academic knowledge graphs.<n>AISE-Bench release contains 1,133 QA pairs, including query, full API execution, validated parameters, and source-grounded answers with reference links.<n>We develop a comprehensive evaluation protocol measuring answer quality, reference grounding, API-planning correctness, and execution success.
  arXiv  Detail & Related papers  (2026-06-16T07:59:46Z)
• DeepAgent: A General Reasoning Agent with Scalable Toolsets [111.6384541877723]
  DeepAgent is an end-to-end deep reasoning agent that performs autonomous thinking, tool discovery, and action execution.<n>To address the challenges of long-horizon interactions, we introduce an autonomous memory folding mechanism that compresses past interactions into structured episodic, working, and tool memories.<n>We develop an end-to-end reinforcement learning strategy, namely ToolPO, that leverages LLM-simulated APIs and applies tool-call advantage attribution to assign fine-grained credit to the tool invocation tokens.
  arXiv  Detail & Related papers  (2025-10-24T16:24:01Z)
• In-N-Out: A Parameter-Level API Graph Dataset for Tool Agents [12.78469884522289]
  We introduce In-N-Out, the first expert-annotated dataset of API graphs built from two real-world API benchmarks and their documentation.<n>Using In-N-Out significantly improves performance on both tool retrieval and multi-tool query generation.<n>Our findings highlight the promise of using explicit API graphs for tool agents and the utility of In-N-Out as a valuable resource.
  arXiv  Detail & Related papers  (2025-09-01T15:42:21Z)
• OASBuilder: Generating OpenAPI Specifications from Online API Documentation with Large Language Models [10.54692787937075]
  OASBuilder is a framework that transforms long and diverse API documentation pages into consistent, machine-readable API specifications.<n> OASBuilder has been successfully implemented in an enterprise environment, saving thousands of hours of manual effort.
  arXiv  Detail & Related papers  (2025-07-07T14:36:13Z)
• A Framework for Testing and Adapting REST APIs as LLM Tools [5.758488787763118]
  We present a novel testing framework aimed at evaluating and enhancing the readiness of REST APIs to function as tools for agents.<n>Our framework transforms apis as tools, generates comprehensive test cases for the APIs, tests cases into natural language instructions and evaluates the agent's ability t correctly invoke the API and process its inputs and responses.
  arXiv  Detail & Related papers  (2025-04-22T02:52:08Z)
• ToolFactory: Automating Tool Generation by Leveraging LLM to Understand REST API Documentations [4.934192277899036]
  API documentation often suffers from a lack of standardization, inconsistent schemas, and incomplete information.<n>We developed textbfToolFactory, an open-source pipeline for automating tool generation from unstructured API documents.<n>We also demonstrated ToolFactory by creating a domain-specific AI agent for glycomaterials research.
  arXiv  Detail & Related papers  (2025-01-28T13:42:33Z)
• A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs [46.65963514391019]
  We present AutoRestTest, the first black-box tool to adopt a dependency-embedded multi-agent approach for REST API testing.<n>Our approach treats REST API testing as a separable problem, where four agents collaborate to optimize API exploration.<n>Our evaluation of AutoRestTest on 12 real-world REST services shows that it outperforms the four leading black-box REST API testing tools.
  arXiv  Detail & Related papers  (2024-11-11T16:20:27Z)
• Beyond Browsing: API-Based Web Agents [58.39129004543844]
  API-Based Agents outperform web Browsing Agents in experiments on WebArena.<n>Hybrid Agents out-perform both others nearly uniformly across tasks.<n>Results strongly suggest that when APIs are available, they present an attractive alternative to relying on web browsing alone.
  arXiv  Detail & Related papers  (2024-10-21T19:46:06Z)
• A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How [53.65636914757381]
  API suggestion is a critical task in modern software development. Recent advancements in large code models (LCMs) have shown promise in the API suggestion task.
  arXiv  Detail & Related papers  (2024-09-20T03:12:35Z)
• WorldAPIs: The World Is Worth How Many APIs? A Thought Experiment [49.00213183302225]
  We propose a framework to induce new APIs by grounding wikiHow instruction to situated agent policies.<n>Inspired by recent successes in large language models (LLMs) for embodied planning, we propose a few-shot prompting to steer GPT-4.
  arXiv  Detail & Related papers  (2024-07-10T15:52:44Z)
• DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model [90.71963723884944]
  Text-to-image (T2I) generative models have attracted significant attention and found extensive applications within and beyond academic research. We introduce DiffAgent, an agent designed to screen the accurate selection in seconds via API calls. Our evaluations reveal that DiffAgent not only excels in identifying the appropriate T2I API but also underscores the effectiveness of the SFTA training framework.
  arXiv  Detail & Related papers  (2024-03-31T06:28:15Z)
• You Can REST Now: Automated REST API Documentation and Testing via LLM-Assisted Request Mutations [8.158964648211002]
  We present RESTSpecIT, the first automated approach that infers documentation and performs black-box testing of REST APIs.<n>Our approach requires minimal user input compared to state-of-the-art tools.<n>We evaluate the quality of our tool with three state-of-the-art LLMs: DeepSeek V3, GPT-4.1, and GPT-3.5.
  arXiv  Detail & Related papers  (2024-02-07T18:55:41Z)
• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs [104.37772295581088]
  Open-source large language models (LLMs), e.g., LLaMA, remain significantly limited in tool-use capabilities. We introduce ToolLLM, a general tool-usetuning encompassing data construction, model training, and evaluation. We first present ToolBench, an instruction-tuning framework for tool use, which is constructed automatically using ChatGPT.
  arXiv  Detail & Related papers  (2023-07-31T15:56:53Z)
• Carving UI Tests to Generate API Tests and API Specification [8.743426215048451]
  API-level testing can play an important role, in-between unit-level testing and UI-level (or end-to-end) testing. Existing API testing tools require API specifications, which often may not be available or, when available, be inconsistent with the API implementation. We present an approach that leverages UI testing to enable API-level testing for web applications.
  arXiv  Detail & Related papers  (2023-05-24T03:53:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.