Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents

• URL: http://arxiv.org/abs/2602.16943v1
• Date: Wed, 18 Feb 2026 23:17:15 GMT
• Title: Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents
• Abstract summary: GAP benchmark measures divergence between text-level safety and tool-call-level safety in LLM agents.<n>We observe instances where the model's text output refuses a harmful request while its tool calls simultaneously execute the forbidden action--a divergence we formalize as the GAP metric.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models deployed as agents increasingly interact with external systems through tool calls--actions with real-world consequences that text outputs alone do not carry. Safety evaluations, however, overwhelmingly measure text-level refusal behavior, leaving a critical question unanswered: does alignment that suppresses harmful text also suppress harmful actions? We introduce the GAP benchmark, a systematic evaluation framework that measures divergence between text-level safety and tool-call-level safety in LLM agents. We test six frontier models across six regulated domains (pharmaceutical, financial, educational, employment, legal, and infrastructure), seven jailbreak scenarios per domain, three system prompt conditions (neutral, safety-reinforced, and tool-encouraging), and two prompt variants, producing 17,420 analysis-ready datapoints. Our central finding is that text safety does not transfer to tool-call safety. Across all six models, we observe instances where the model's text output refuses a harmful request while its tool calls simultaneously execute the forbidden action--a divergence we formalize as the GAP metric. Even under safety-reinforced system prompts, 219 such cases persist across all six models. System prompt wording exerts substantial influence on tool-call behavior: TC-safe rates span 21 percentage points for the most robust model and 57 for the most prompt-sensitive, with 16 of 18 pairwise ablation comparisons remaining significant after Bonferroni correction. Runtime governance contracts reduce information leakage in all six models but produce no detectable deterrent effect on forbidden tool-call attempts themselves. These results demonstrate that text-only safety evaluations are insufficient for assessing agent behavior and that tool-call safety requires dedicated measurement and mitigation.

Related papers

• BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents [20.217145643468598]
  We introduce Blindspot, a benchmark for trajectory-level safety calibration of long-horizon tool-using agents.<n>Blindspot evaluates complete user-agent-environment trajectories through adaptive adversarial interaction, stateful tool execution, and execution-grounded adjudication.<n>We evaluate 13 proprietary and open-weight LLMs using eight metrics covering unsafe completion, appropriate refusal, benign utility, over-refusal, repeated-run robustness, and post-refusal failure.
  arXiv  Detail & Related papers  (2026-09-14T20:12:27Z)
• EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents [74.54929751174289]
  EvoSafeHarness is a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain.<n>It reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost.<n>It also achieves the best score on Agent-SafetyBench for every victim.
  arXiv  Detail & Related papers  (2026-09-05T05:56:41Z)
• Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents [21.622196229181103]
  We identify schema-formatted tool specifications as a primary source of agent safety degradation.<n>We propose SafeKeep, an inference-time safeguard that decouples safety judgment from tool execution.<n>It increases the average refusal rate for harmful requests from 23.8% to 70.6% and reduces the average attack success rate under observation-level prompt injection from 25.6% to 2.5%.
  arXiv  Detail & Related papers  (2026-07-31T10:25:04Z)
• What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants [4.032084095153604]
  We screen 68,816 papers from 22 premier venues, 185 curating safety-relevant studies, and mine 16,586 GitHub issues from widely deployed coding tools.<n>Applying systematic open coding over both corpora, we derive a multi-dimensional safety taxonomy of 33 operational risk types.<n>Our findings show that coding-agent failures are often severe, with 326 of 547 incidents rated high or critical.
  arXiv  Detail & Related papers  (2026-05-29T03:09:37Z)
• Measuring Safety Alignment Effects in Autonomous Security Agents [3.93181912653522]
  Security agents must inspect repositories, call tools, and produce vulnerability evidence inside authorized sandboxes.<n>We present a trace-based benchmark of 30 local vulnerability-analysis tasks with fixed tools, deterministic success predicates, redaction rules, and grounding checks.<n>We show that safety alignment effects in autonomous security agents should be measured at the system level, separating refusal, unsafe action, tool reliability, and evidence grounding.
  arXiv  Detail & Related papers  (2026-05-19T11:55:54Z)
• Selective Safety Steering via Value-Filtered Decoding [54.87935112120107]
  Large language models (LLMs) are trained to align with human values, but their generations may still violate safety constraints.<n>Existing decoding-time steering methods often intervene unnecessarily, modifying generations that would have been safe under the base model.<n>We propose a new test-time steering method designed to reduce such unnecessary interventions while improving the safety of unsafe responses.
  arXiv  Detail & Related papers  (2026-05-14T12:13:08Z)
• On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignment [54.30690671490447]
  Existing safety-alignment signals are largely response-level or off-policy.<n>FATE transforms verifier-scored failures into repair supervision without expert demonstrations.<n>FATE reduces attack success rate by 33.5%, harmful compliance by 82.6%, and improves external trajectory-safety diagnosis by 6.5%.
  arXiv  Detail & Related papers  (2026-05-12T09:56:28Z)
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation [68.53387633351484]
  We propose CORA (COnformal Risk-controlled GUI Agent), a post-policy, pre-action safeguarding framework that provides statistical guarantees on harmful executed actions.<n> CORA reformulates safety as selective action execution: we train a Guardian model to estimate action-conditional risk for each proposed step.<n>To rigorously evaluate this paradigm, we introduce Phone-Harm, a new benchmark of mobile safety violations with step-level harm labels.
  arXiv  Detail & Related papers  (2026-04-10T09:41:21Z)
• Robust Multimodal Safety via Conditional Decoding [52.92816441364308]
  Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions.<n>We propose a simple conditional decoding strategy, CASA, that utilizes internal representations of MLLMs to predict a binary safety token before response generation.
  arXiv  Detail & Related papers  (2026-03-31T23:19:50Z)
• The Causal Impact of Tool Affordance on Safety Alignment in LLM Agents [0.3823356975862005]
  This work empirically examines how executable tool affordance alters safety alignment in large language models (LLMs)<n>Experiments are conducted in a deterministic financial transaction environment with binary safety constraints across 1,500 scenarios.<n>Both evaluated models maintain perfect compliance in text-only settings, yet exhibit sharp increases in violations after tool access is introduced.
  arXiv  Detail & Related papers  (2026-03-19T23:34:46Z)
• ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback [53.2744585868162]
  Monitoring step-level tool invocation behaviors in real time is critical for agent deployment.<n>We first construct TS-Bench, a novel benchmark for step-level tool invocation safety detection in LLM agents.<n>We then develop a guardrail model, TS-Guard, using multi-task reinforcement learning.
  arXiv  Detail & Related papers  (2026-01-15T07:54:32Z)
• What Matters For Safety Alignment? [38.86339753409445]
  This paper presents a comprehensive empirical study on the safety alignment capabilities of AI systems.<n>We systematically investigate and compare the influence of six critical intrinsic model characteristics and three external attack techniques.<n>We identify the LRMs GPT-OSS-20B, Qwen3-Next-80B-A3B-Thinking, and GPT-OSS-120B as the top-three safest models.
  arXiv  Detail & Related papers  (2026-01-07T12:31:52Z)
• GSAE: Graph-Regularized Sparse Autoencoders for Robust LLM Safety Steering [5.124731939041066]
  Large language models (LLMs) face critical safety challenges as they can be manipulated to generate harmful content through adversarial prompts and jailbreak attacks.<n>We introduce Graph-Regularized Sparse Autoencoders (GSAEs), which extends SAEs with a Laplacian smoothness penalty on the neuron co-activation graph.<n>We demonstrate that GSAE enables effective safety steering, assembling features into a weighted set of safety-relevant directions and controlling them with a two-stage gating mechanism.
  arXiv  Detail & Related papers  (2025-12-07T04:46:30Z)
• Securing the Model Context Protocol: Defending LLMs Against Tool Poisoning and Adversarial Attacks [8.419049623790618]
  This work analyzes three classes of semantic attacks on MCP-integrated systems.<n>We introduce a layered security framework with three components: RSA-based manifest signing to enforce descriptor integrity, LLM-on-LLM semantic vetting to detect suspicious tool definitions, and lightweight guardrails that block anomalous tool behavior at runtime.<n>Our results show that the proposed framework reduces unsafe tool invocation rates without model fine-tuning or internal modification.
  arXiv  Detail & Related papers  (2025-12-06T20:07:58Z)
• Qwen3Guard Technical Report [127.69960525219051]
  We present Qwen3Guard, a series of multilingual safety guardrail models with two specialized variants.<n>Generative Qwen3Guard casts safety classification as an instruction-following task to enable fine-grained tri-class judgments.<n>Stream Qwen3Guard introduces a token-level classification head for real-time safety monitoring.
  arXiv  Detail & Related papers  (2025-10-16T04:00:18Z)
• STAC: When Innocent Tools Form Dangerous Chains to Jailbreak LLM Agents [38.755035623707656]
  This paper introduces Sequential Tool Attack Chaining (STAC), a novel multi-turn attack framework that exploits agent tool use.<n>We apply our framework to automatically generate and evaluate 483 STAC cases, featuring 1,352 sets of user-agent-environment interactions.<n>Our evaluations show that state-of-the-art LLM agents, including GPT-4.1, are highly vulnerable to STAC, with attack success rates (ASR) exceeding 90% in most cases.
  arXiv  Detail & Related papers  (2025-09-30T00:31:44Z)
• CARE: Decoding Time Safety Alignment via Rollback and Introspection Intervention [68.95008546581339]
  Existing decoding-time interventions, such as Contrastive Decoding, often force a severe trade-off between safety and response quality.<n>We propose CARE, a novel framework for decoding-time safety alignment that integrates three key components.<n>The framework achieves a superior balance of safety, quality, and efficiency, attaining a low harmful response rate and minimal disruption to the user experience.
  arXiv  Detail & Related papers  (2025-09-01T04:50:02Z)
• ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning [64.32925552574115]
  ARMOR is a large language model that analyzes jailbreak strategies and extracts the core intent.<n> ARMOR achieves state-of-the-art safety performance, with an average harmful rate of 0.002 and an attack success rate of 0.06 against advanced optimization-based jailbreaks.
  arXiv  Detail & Related papers  (2025-07-14T09:05:54Z)
• SafeDialBench: A Fine-Grained Safety Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attacks [90.41592442792181]
  We propose a fine-grained benchmark SafeDialBench for evaluating the safety of Large Language Models (LLMs)<n>Specifically, we design a two-tier hierarchical safety taxonomy that considers 6 safety dimensions and generates more than 4000 multi-turn dialogues in both Chinese and English under 22 dialogue scenarios.<n> Notably, we construct an innovative assessment framework of LLMs, measuring capabilities in detecting, and handling unsafe information and maintaining consistency when facing jailbreak attacks.
  arXiv  Detail & Related papers  (2025-02-16T12:08:08Z)
• SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering [56.92068213969036]
  Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions.<n>Recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue.<n>We propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns.
  arXiv  Detail & Related papers  (2024-08-21T10:01:34Z)
• Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training [67.30423823744506]
  We introduce a novel approach, Decoupled Refusal Training (DeRTa), designed to empower LLMs to refuse compliance to harmful prompts at any response position.<n>DeRTa incorporates two novel components: (1) Maximum Likelihood Estimation with Harmful Response Prefix, which trains models to recognize and avoid unsafe content by appending a segment of harmful response to the beginning of a safe response, and (2) Reinforced Transition Optimization (RTO), which equips models with the ability to transition from potential harm to safety refusal consistently throughout the harmful response sequence.
  arXiv  Detail & Related papers  (2024-07-12T09:36:33Z)
• On Prompt-Driven Safeguarding for Large Language Models [172.13943777203377]
  We find that in the representation space, the input queries are typically moved by safety prompts in a "higher-refusal" direction. Inspired by these findings, we propose a method for safety prompt optimization, namely DRO. Treating a safety prompt as continuous, trainable embeddings, DRO learns to move the queries' representations along or opposite the refusal direction, depending on their harmfulness.
  arXiv  Detail & Related papers  (2024-01-31T17:28:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.