Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoning

• URL: http://arxiv.org/abs/2601.07641v1
• Date: Mon, 12 Jan 2026 15:22:51 GMT
• Title: Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoning
• Abstract summary: Test-Time Tool Evolution (TTE) is a new paradigm that enables agents to synthesize, verify, and evolve executable tools during inference.<n>By transforming tools from fixed resources into problem-driven artifacts, TTE overcomes the rigidity and long-tail limitations of static tool libraries.
• Score: 28.61961520286559
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The central challenge of AI for Science is not reasoning alone, but the ability to create computational methods in an open-ended scientific world. Existing LLM-based agents rely on static, pre-defined tool libraries, a paradigm that fundamentally fails in scientific domains where tools are sparse, heterogeneous, and intrinsically incomplete. In this paper, we propose Test-Time Tool Evolution (TTE), a new paradigm that enables agents to synthesize, verify, and evolve executable tools during inference. By transforming tools from fixed resources into problem-driven artifacts, TTE overcomes the rigidity and long-tail limitations of static tool libraries. To facilitate rigorous evaluation, we introduce SciEvo, a benchmark comprising 1,590 scientific reasoning tasks supported by 925 automatically evolved tools. Extensive experiments show that TTE achieves state-of-the-art performance in both accuracy and tool efficiency, while enabling effective cross-domain adaptation of computational tools. The code and benchmark have been released at https://github.com/lujiaxuan0520/Test-Time-Tool-Evol.

Related papers

• SciToolAgent-Evo: An Ontology-Aware Self-Evolving Agent for Open-World Scientific Tool Acquisition [53.36281538053445]
  We propose SciToolAgent-Evo, an ontology-aware self-evolving agent for open-world scientific tool acquisition.<n> Driven by an evolving memory of skills, experiences, and an ontologized tool graph, it distills generalizable knowledge from contrastive trajectories during accumulation.<n>Once a novel tool is acquired, its scientific knowledge is completed online for seamless integration into the known graph.
  arXiv  Detail & Related papers  (2026-07-30T08:47:11Z)
• Tool-Genesis: A Task-Driven Tool Creation Benchmark for Self-Evolving Language Agent [45.450766613995135]
  Tool-Genesis is a diagnostic benchmark designed to quantify agent capabilities across multiple dimensions.<n>We find that even state-of-the-art models struggle to produce precise tool interfaces or executable logic in a one-shot setting.
  arXiv  Detail & Related papers  (2026-03-05T17:44:29Z)
• SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents [100.12367115920121]
  We introduce SciGymAgent, a scalable interactive environment featuring 1,780 domain-specific tools across four natural science disciplines.<n>We also present SciAgentBench, a tiered evaluation suite designed to stress-test agentic capabilities.
  arXiv  Detail & Related papers  (2026-02-13T14:58:18Z)
• Evolving from Tool User to Creator via Training-Free Experience Reuse in Multimodal Reasoning [16.12114923351562]
  We propose a training-free framework that transforms agents from tool users to tool creators.<n>This approach harvests reasoning experiences and distills them into reusable assets.<n>We also introduce a memory consolidation mechanism to maintain the tool library.
  arXiv  Detail & Related papers  (2026-02-02T11:37:45Z)
• Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning [63.071280297939005]
  We present Transductive Visual Programming (TVP), a novel framework that builds new tools from its own experience rather than speculation.<n>TVP achieves state-of-the-art performance, outperforming GPT-4o by 22% and the previous best visual programming system by 11%.<n>Our work establishes experience-driven transductive tool creation as a powerful paradigm for building self-evolving visual programming agents.
  arXiv  Detail & Related papers  (2025-12-24T04:30:21Z)
• An Agentic Framework for Autonomous Materials Computation [70.24472585135929]
  Large Language Models (LLMs) have emerged as powerful tools for accelerating scientific discovery.<n>Recent advances integrate LLMs into agentic frameworks, enabling retrieval, reasoning, and tool use for complex scientific experiments.<n>Here, we present a domain-specialized agent designed for reliable automation of first-principles materials computations.
  arXiv  Detail & Related papers  (2025-12-22T15:03:57Z)
• SynthTools: A Framework for Scaling Synthetic Tools for Agent Development [5.411691071249542]
  We introduce SynthTools, a flexible and scalable framework for generating synthetic tool ecosystems.<n>Our framework consists of three core components: Tool Generation for automatic and scalable creation of diverse tools, Tool Simulation to emulate realistic tool behaviors, and Tool Audit to ensure correctness and consistency of tool simulation.<n>By enabling scalable, diverse, and reliable tool ecosystems, SynthTools provides a practical path toward large-scale training and stable evaluation of tool-use agents.
  arXiv  Detail & Related papers  (2025-11-11T19:26:40Z)
• MCPVerse: An Expansive, Real-World Benchmark for Agentic Tool Use [72.53177559476704]
  We introduce MCPVerse, a real-world benchmark for evaluating agentic tool use.<n> MCPVerse integrates more than 550 real-world, executable tools to create an unprecedented action space exceeding 140k tokens.<n>We benchmarked the state-of-the-art LLMs across three modes (Oracle, Standard, and Max-Scale)
  arXiv  Detail & Related papers  (2025-08-22T09:47:53Z)
• SciToolAgent: A Knowledge Graph-Driven Scientific Agent for Multi-Tool Integration [39.43814195462455]
  SciToolAgent automates hundreds of scientific tools across biology, chemistry, and materials science.<n>The agent also incorporates a comprehensive safety-checking module to ensure responsible and ethical tool usage.
  arXiv  Detail & Related papers  (2025-07-27T13:55:35Z)
• Acting Less is Reasoning More! Teaching Model to Act Efficiently [87.28134636548705]
  Tool-integrated reasoning augments large language models with the ability to invoke external tools to solve tasks.<n>Current approaches typically optimize only for final correctness without considering the efficiency or necessity of external tool use.<n>We propose a framework that encourages models to produce accurate answers with minimal tool calls.<n>Our approach reduces tool calls by up to 68.3% and improves tool productivity by up to 215.4%, while maintaining comparable answer accuracy.
  arXiv  Detail & Related papers  (2025-04-21T05:40:05Z)
• LLM Agents Making Agent Tools [2.5529148902034637]
  Tool use has turned large language models (LLMs) into powerful agents that can perform complex multi-step tasks.<n>But these tools must be implemented in advance by human developers.<n>We propose ToolMaker, an agentic framework that autonomously transforms papers with code into LLM-compatible tools.
  arXiv  Detail & Related papers  (2025-02-17T11:44:11Z)
• ToolGen: Unified Tool Retrieval and Calling via Generation [34.34787641393914]
  We introduce ToolGen, a paradigm shift that integrates tool knowledge directly into the large language models' parameters.<n>We show that ToolGen achieves superior results in both tool retrieval and autonomous task completion.<n>ToolGen paves the way for more versatile, efficient, and autonomous AI systems.
  arXiv  Detail & Related papers  (2024-10-04T13:52:32Z)
• SciAgent: Tool-augmented Language Models for Scientific Reasoning [129.51442677710452]
  We introduce a new task setting named tool-augmented scientific reasoning. This setting supplements Large Language Models with scalable toolsets. We construct a tool-augmented training corpus named MathFunc which encompasses over 30,000 samples and roughly 6,000 tools. Building on MathFunc, we develop SciAgent to retrieve, understand and, if necessary, use tools for scientific problem solving.
  arXiv  Detail & Related papers  (2024-02-18T04:19:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.