Tool Calling is Linearly Readable and Steerable in Language Models

• URL: http://arxiv.org/abs/2605.07990v1
• Date: Fri, 08 May 2026 16:47:08 GMT
• Title: Tool Calling is Linearly Readable and Steerable in Language Models
• Abstract summary: When a tool-calling agent picks the wrong tool, the failure is invisible until execution.<n>We use 12 instruction-tuned models across Gemma 3, Qwen 3, Qwen 2.5, and Llama 3.1.<n>We find the identity of the chosen tool is linearly and steerable inside the model.
• Score: 7.851713181228514
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When a tool-calling agent picks the wrong tool, the failure is invisible until execution: the email gets sent, the meeting gets missed. Probing 12 instruction-tuned models across Gemma 3, Qwen 3, Qwen 2.5, and Llama 3.1 (270M to 27B), we find the identity of the chosen tool is linearly readable and steerable inside the model. Adding the mean-difference between two tools' average internal activations switches which tool the model selects at 77-100% accuracy on name-only single-turn prompts (93-100% at 4B+), and the JSON arguments that follow autoregressively match the new tool's schema, so flipping the name is enough. The same per-tool means also flag likely errors before they happen: on Gemma 3 12B and 27B, queries where the gap between the top-1 and top-2 tool is smallest produce 14-21x more wrong calls than queries with the largest gap. The causal effect concentrates along one direction, the row of the output layer that produces the target tool's first token: a unit vector along it at matched magnitude already reaches 93-100%, while what is left over leaves the choice almost untouched. Activation patching localises this to a small set of mid- and late-layer attention heads, and a within-topic probe across 14 same-domain $τ$-bench airline tools reaches top-1 61-89% across five 4B-14B models, ruling out the reading that we are just moving the model along a topic axis. Even base models encode the right tool before they can emit it: cosine readout from the internal state recovers 69-82% on BFCL while base generation reaches only 2-10%, suggesting pretraining forms the representation and instruction tuning later wires it to the output. We measure tool identity selection and JSON schema correctness in single-turn fixed-menu settings; multi-turn agentic transfer is more fragile and is discussed in Limitations.

Related papers

• Interface-Induced Trajectory Censoring [6.496285418517832]
  We report a tool-call rate read off the serving stack.<n>That number can be zero while the model is emitting well-formed calls.<n>We reproduce the funnel across a 21x scale range of Qwen2.5-Coder.
  arXiv  Detail & Related papers  (2026-09-03T15:00:31Z)
• Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling [65.03665015319457]
  We propose an action-class-oriented diagnostic framework that decomposes multi-turn failures into two modes: action-class miscalibration and action-execution failure.<n>We validate it on a panel of tool-calling models across multiple multi-turn benchmarks.
  arXiv  Detail & Related papers  (2026-09-01T09:08:15Z)
• Joint Optimization of Tool Creation and Use for Large Language Model Agents [69.07066297088187]
  SMITH (-grounded Multi-task Iterative Tool Honing) trains tool creation and tool use inside a single policy.<n>A 4B Qwen3 trained with SMITH on 13 procedural reasoning tasks with exact verifiers reaches 79.8 macro-average accuracy on held-out tasks.<n>It also reaches 40.4 on TabMWP-Hard and 42.6 on out-of-domain GQA (+7.6 over the best same-backbone inference-time baseline)
  arXiv  Detail & Related papers  (2026-08-25T13:59:34Z)
• ToolFailBench: Diagnosing Tool-Use Failures in LLM Agents [0.0]
  ToolFailBench is a diagnostic benchmark for measuring tool-use failures.<n>We label each trace with Tool-Skip, Result-Ignore, Output-Fabrication, and Unnecessary-Tool-Use.
  arXiv  Detail & Related papers  (2026-07-06T05:25:21Z)
• Looking Is Not Picking: An Attention-Segment Account of Tool-Selection Failures in LLM Agents [0.4855342112648282]
  LLM agents mis-call tools, and the natural guess is that the model failed to see the right tool in a crowded harness.<n>We show the opposite through a lens concurrent work sets aside -- the model's attention to labeled tool-definition segments.
  arXiv  Detail & Related papers  (2026-06-15T07:58:56Z)
• When the Tool Decides: LLM Agents Defer Blindly to Graph Neural Network Tools, and Stronger Backbones Defer More [10.283803346813047]
  A growing line of work equips large language model (LLM) agents with graph neural networks (GNNs) as callable tools.<n>We expose a frozen GNN to a ReAct-style LLM agent as an explicit tool and measure, on node classification over a text-attributed graph.<n>We find the agent does not exercise judgment: its predictions agree with the raw GNN's 97.6-99.2% of the time (5 seeds)<n>We find the cost of deference does not shrink as capability grows and grows where alternatives emerge.
  arXiv  Detail & Related papers  (2026-06-12T14:13:19Z)
• HyperTool: Beyond Step-Wise Tool Calls for Tool-Augmented Agents [52.33697136670754]
  We introduce textbfHyperTool, a unified executable MCP-style tool interface that changes the model-visible unit of tool execution.<n>A model invokes HyperTool with a code block that can call existing tools through their original schemas, manipulate returned values, and pass intermediate results locally.<n>On MCP-Universe, HyperTool improves average accuracy from 15.69% to 35.29% on Qwen3-32B and from 9.93% to 33.33% on Qwen3-8B, and surpass GPT-OSS and Kimi-k2.5 on average
  arXiv  Detail & Related papers  (2026-06-11T17:56:36Z)
• Agent Explorative Policy Optimization for Multimodal Agentic Reasoning [97.64835302176056]
  Agentic reasoning interleaves two behaviors with a structural asymmetry: thinking and tool use.<n>Under standard RL recipes like GRPO, the gap manifests as two diagnostic symptoms during training.<n>We propose AXPO (Agent eXplorative Policy Optimization) for each all-wrong tool-using subgroup.
  arXiv  Detail & Related papers  (2026-05-27T17:36:39Z)
• Measuring Maximum Activations in Open Large Language Models [60.3514350516308]
  We measure global and layerwise maxima on 27 checkpoints from 8 open families spanning dense, MoE, vision-language, intermediate-training, and instruction-tuned variants.<n>We conclude that maximum activation magnitude is a model property tied to family, architecture, and training stage - not a simple byproduct of size.
  arXiv  Detail & Related papers  (2026-05-15T03:31:51Z)
• ARISE: A Repository-level Graph Representation and Toolset for Agentic Fault Localization and Program Repair [0.0]
  We present ARISE (Agentic Repository-level Issue Solving Engine), which augments an LLM-based agent with a multi-granularity program graph.<n>ARISE exposes this graph through a three-tier tool API, which brings data-flow slicing as a first-class, queryable agent primitive.<n>We evaluate on SWE-bench Lite (300 real GitHub issues, 11 Python repositories) using Qwen2.5-Coder-32B-Instruct as the backbone.
  arXiv  Detail & Related papers  (2026-05-04T19:59:23Z)
• Squeez: Task-Conditioned Tool-Output Pruning for Coding Agents [0.8714519393367773]
  Coding agents repeatedly consume long tool observations even though only a small fraction of each observation matters for the next step.<n>We study task-conditioned tool-output pruning: given a focused query and one tool output, return the smallest verbatim evidence block the agent should inspect next.
  arXiv  Detail & Related papers  (2026-04-04T18:52:44Z)
• AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents [50.481033105867205]
  We introduce AgentProcessBench, the first benchmark dedicated to evaluating step-level effectiveness in realistic, tool-augmented trajectories.<n>The benchmark comprises 1,000 diverse trajectories and 8,509 human-labeled step annotations with 89.1% inter-annotator agreement.<n>It features a ternary labeling scheme to capture exploration and an error propagation rule to reduce labeling ambiguity.
  arXiv  Detail & Related papers  (2026-03-15T16:13:58Z)
• ToolFlood: Beyond Selection -- Hiding Valid Tools from LLM Agents via Semantic Covering [2.6928305857508974]
  This paper introduces ToolFlood, a retrieval-layer attack on tool-augmented Large Language Model (LLM) agents.<n>Rather than altering which tool is chosen after retrieval, ToolFlood overwhelms retrieval itself by injecting a few attacker-controlled tools.<n>ToolFlood achieves up to a 95% attack success rate with a low injection rate.
  arXiv  Detail & Related papers  (2026-03-14T13:54:49Z)
• AlphaApollo: Orchestrating Foundation Models and Professional Tools into a Self-Evolving System for Deep Agentic Reasoning [110.57865233597762]
  We present AlphaApollo, a self-evolving agentic reasoning system.<n>It aims to address two bottlenecks in foundation model (FM) reasoning-limited model-intrinsic capacity and unreliable test-time computation.<n>AlphaApollo orchestrates multiple models with professional tools to enable deliberate, verifiable reasoning.
  arXiv  Detail & Related papers  (2025-10-05T15:42:24Z)
• MCP-Zero: Active Tool Discovery for Autonomous LLM Agents [13.005899769943442]
  We introduce MCP-Zero, an active agent framework that restores tool discovery autonomy to LLMs themselves.<n>Instead of overwhelming models with all available tools, MCP-Zero enables agents to actively identify capability gaps, and request specific tools on-demand.<n>We construct MCP-tools, a comprehensive dataset of 308 MCP servers and 2,797 tools from the official Model-Context-Protocol repository.
  arXiv  Detail & Related papers  (2025-06-01T15:48:53Z)
• A Simple Linear Patch Revives Layer-Pruned Large Language Models [58.056251480151104]
  Layer pruning has emerged as a widely used technique for compressing large language models (LLMs)<n>textscLinearPatch fuses two operations into one matrix multiply at the pruning interface.<n>The patch can be further refined with 5K unlabeled samples via memory-efficient offline distillation, pushing the retention to 95.16% within only 30 minutes on a single GPU.
  arXiv  Detail & Related papers  (2025-05-30T15:06:08Z)
• S*: Test Time Scaling for Code Generation [55.11863577956177]
  We propose S*, the first hybrid test-time scaling framework for code generation.<n>S* substantially improves the coverage and selection accuracy of generated code.
  arXiv  Detail & Related papers  (2025-02-20T09:18:53Z)
• s1: Simple test-time scaling [148.4204982041058]
  Test-time scaling is a promising new approach to language modeling that uses extra test-time compute to improve performance.<n>We seek the simplest approach to achieve test-time scaling and strong reasoning performance.
  arXiv  Detail & Related papers  (2025-01-31T18:48:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.