ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs

• URL: http://arxiv.org/abs/2607.14285v1
• Date: Wed, 15 Jul 2026 18:48:49 GMT
• Title: ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs
• Abstract summary: We find that safety-aligned open-source models override their deployment instructions up to 43.4% of the time.<n>We also find that abliteration reduces rates of external whistleblowing.<n>These results reveal a fundamental tension in pluralistic alignment, where the same safety training that protects users can cause agents to act against deployment instructions in ways that create unpredictable liability risks.
• Score: 11.599619860803472
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safety alignment in LLMs aims to align models with human values, but which values take precedence when they conflict? We investigate this question in the context of tool-calling LLM agents deployed in regulated industries, where agents processing confidential documents may encounter content that triggers safety-trained values (e.g., public welfare) that conflict with deployment-context instructions (e.g., internal logging). To empirically verify this phenomenon, we build a benchmark of 128 scenarios across 16 domains. We find that safety-aligned open-source models override their deployment instructions up to 43.4% of the time, engaging in whistleblowing, data exfiltration, and evidence tampering when processing documents that suggest organizational wrongdoing. We also find that abliteration reduces rates of external whistleblowing. These results reveal a fundamental tension in pluralistic alignment, where the same safety training that protects users can cause agents to act against deployment instructions in ways that create unpredictable liability risks. We release our benchmark as a framework to support evaluation of agent behavior under competing legitimate interests.

Related papers

• Adversarial Attacks in Multi-Agent LLM Pipelines: Unveiling Structural Vulnerabilities in Agentic AI Architectures [4.053243845971823]
  Multi-agent LLM pipelines orchestrate specialized language model agents into structured where intermediate outputs are passed across agents to solve complex tasks.<n>This design introduces a security gap absent in single-agent settings: once an agent accepts adversarial content, it is propagated as trusted input throughout the pipeline.<n>We argue that this vulnerability stems from the absence of boundary verification, a security primitive that enforces explicit validation of data as it crosses inter-agent boundaries.
  arXiv  Detail & Related papers  (2026-08-01T15:35:58Z)
• AgentLens: Interpretable Safety Steering via Mechanistic Subspaces for Multi-Turn Coding Agent [53.82005364479556]
  Coding agents based on large language models (LLMs) demonstrate remarkable autonomous capabilities.<n>Existing safety mechanisms mainly rely on external guardrails.<n>We propose AgentLens, a white-box defense framework that performs runtime safety detection and representation-level mitigation.
  arXiv  Detail & Related papers  (2026-06-21T21:13:55Z)
• Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation [13.654207263751156]
  Large language model (LLM) agents are rapidly moving from conversational interfaces to software components that plan, invoke tools, maintain memory, and act on external environments.<n>This paper synthesizes 247 papers through a lifecycle-based, systems-oriented framework that models agent security around the interaction of information flow, delegated authority, and persistent state.<n>We find that prompt injection and tool-mediated control-flow hijacking still dominate the field, while persistent state corruption and multi-agent propagation are becoming central emerging concerns.
  arXiv  Detail & Related papers  (2026-06-09T12:01:07Z)
• Steering LLM Viewpoints through Fabricated Evidence Injection [61.22850755490993]
  We introduce a two-phase attack framework that first repackages misleading statements with fabricated rationales, then instruct target LLMs to incorporate these viewpoints when responding to relevant queries.<n>Experiments on BBQ, ToxiGen, and our specialized dataset reveal that commercial LLMs without external safety classifiers remain highly vulnerable.<n>Building on this, we explore multiple defense strategies, among which a tailored safety policy enablesgpt-oss-safeguard to achieve 81% detection rate.
  arXiv  Detail & Related papers  (2026-06-04T14:48:23Z)
• Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agents [2.905751301655124]
  We introduce AgentREVEAL, a diagnostic framework for analyzing retrieval-induced safety degradation in LLM agents.<n>We show that even oppositional or safety-oriented sources, such as pages containing warnings or risk disclaimers, can increase harmful compliance by an average of 25%.<n>Because relevance is also what makes retrieval useful, these results expose a safety-utility trade-off for retrieval-enabled agents.
  arXiv  Detail & Related papers  (2026-05-28T01:23:48Z)
• Securing LLM Agents Need Intent-to-Execution Integrity [49.490963596514185]
  We argue that securing LLM agents requires defining an end-to-end correctness property that specifies when an agent's execution faithfully reflects the user's intent.<n>We observe that LLM agents are structurally analogous to compilers, where security violations correspond to mis-executions that do not preserve user intent.<n>We derive four integrity properties that must hold simultaneously: emphTool Integrity, emphInstruction Integrity, emphJudgment Integrity, and emphData Flow Integrity.
  arXiv  Detail & Related papers  (2026-05-16T12:53:31Z)
• The Causal Impact of Tool Affordance on Safety Alignment in LLM Agents [0.3823356975862005]
  This work empirically examines how executable tool affordance alters safety alignment in large language models (LLMs)<n>Experiments are conducted in a deterministic financial transaction environment with binary safety constraints across 1,500 scenarios.<n>Both evaluated models maintain perfect compliance in text-only settings, yet exhibit sharp increases in violations after tool access is introduced.
  arXiv  Detail & Related papers  (2026-03-19T23:34:46Z)
• You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents [9.719776777345364]
  High-privilege LLM agents that autonomously process external documentation are increasingly trusted to automate tasks.<n>These agents are granted terminal access, control, and outbound network connectivity with minimal security oversight.<n>We measure a fundamental vulnerability in this trust model, which we term the emphTrusted Executor Dilemma<n>This vulnerability is a structural consequence of the instruction-following design paradigm, not an implementation bug.
  arXiv  Detail & Related papers  (2026-03-12T12:35:46Z)
• ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback [53.2744585868162]
  Monitoring step-level tool invocation behaviors in real time is critical for agent deployment.<n>We first construct TS-Bench, a novel benchmark for step-level tool invocation safety detection in LLM agents.<n>We then develop a guardrail model, TS-Guard, using multi-task reinforcement learning.
  arXiv  Detail & Related papers  (2026-01-15T07:54:32Z)
• Towards Verifiably Safe Tool Use for LLM Agents [53.55621104327779]
  Large language model (LLM)-based AI agents extend capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents.<n>LLMs may invoke unintended tool interactions and introduce risks, such as leaking sensitive data or overwriting critical records.<n>Current approaches to mitigate these risks, such as model-based safeguards, enhance agents' reliability but cannot guarantee system safety.
  arXiv  Detail & Related papers  (2026-01-12T21:31:38Z)
• Enterprise AI Must Enforce Participant-Aware Access Control [9.68210477539956]
  Large language models (LLMs) are increasingly deployed in enterprise settings where they interact with multiple users and are trained or fine-tuned on sensitive internal data.<n>We show that adversaries can exploit current fine-tuning and RAG architectures to leak sensitive information by leveraging the lack of access control enforcement.<n>We introduce a framework centered on the principle that any content used in training, retrieval, or generation by an LLM is explicitly authorized for emphall users involved in the interaction.
  arXiv  Detail & Related papers  (2025-09-18T04:30:49Z)
• AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agents [48.925168866726814]
  AgentAuditor is a universal, training-free, memory-augmented reasoning framework.<n>ASSEBench is the first benchmark designed to check how well LLM-based evaluators can spot both safety risks and security threats.
  arXiv  Detail & Related papers  (2025-05-31T17:10:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.