Think Twice Before You Act: Protecting LLM Agents Against Tool Description Poisoning via Isolated Planning

• URL: http://arxiv.org/abs/2606.20922v1
• Date: Thu, 18 Jun 2026 20:31:08 GMT
• Title: Think Twice Before You Act: Protecting LLM Agents Against Tool Description Poisoning via Isolated Planning
• Abstract summary: Cross-tool description poisoning can manipulate planner-visible tool metadata to steer an agent's trajectory.<n>A key observation is that poisoned descriptions persist in the planning context across steps, enabling continuous influence over subsequent tool choices.<n>We propose Tool-Guard, a novel system-level defense based on a new concept called isolated planning.
• Score: 48.99783163807576
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The integration of external tools has substantially expanded the capabilities of large language model (LLM) agents, but it also introduces new attack surfaces beyond prompt injection. In particular, cross-tool description poisoning can manipulate planner-visible tool metadata to steer an agent's trajectory, even if the poisoned tool itself is never chosen. To understand the effectiveness of existing defenses against this emerging threat, we first evaluate several prompt-injection defenses and find that they transfer poorly to cross-tool description poisoning. A key observation is that poisoned descriptions persist in the planning context across steps, enabling continuous influence over subsequent tool choices. Building on this insight, we propose Tool-Guard, a novel system-level defense based on a new concept called isolated planning, in which tool invocations that are detected as misaligned or suspicious cause the corresponding tool to be placed in a quarantined list (the influenced list), breaking further influence from poisoned descriptions. With this influence isolated, the tool can continue to be used to support the task, enabling a robust defense that preserves legitimate tool utility. Experiments on the AgentDojo and ASB benchmarks show that Tool-Guard substantially reduces attack success while maintaining high task utility. Our code is available at https://github.com/shishishi123/Tool-Guard.

Related papers

• ActGuard: Pre-execution Action Auditing against Indirect Prompt Injection in LLM Agents [11.495797074855703]
  We propose ActGuard, a pre-execution action auditing framework.<n>Rather than judging whether external content is inherently suspicious, ActGuard assesses whether it causes the current action to deviate from a reasonable expectation.<n>A verifier then examines the localized evidence, masks only spans confirmed as malicious, and regenerates the action from the sanitized context.
  arXiv  Detail & Related papers  (2026-09-14T03:47:05Z)
• ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection [61.4148196085256]
  We present ROPE (Routed Origin Policy Enforcement), which is anchored in a structural notion of trust.<n>Our approach admits two provable guarantees: 1) at every step of a trajectory, no value whose only origin is attacker-writable content reaches an origin-guarded parameter, and 2) no rewording of an injection changes an admission decision.
  arXiv  Detail & Related papers  (2026-08-27T01:22:14Z)
• TRUSTDESC: Preventing Tool Poisoning in LLM Applications via Trusted Description Generation [16.85010496624083]
  Large language models (LLMs) increasingly rely on external tools to perform time-sensitive tasks and real-world actions.<n>Attackers manipulate tool descriptions by embedding malicious instructions (explicit TPAs) or misleading claims (implicit TPAs) to influence model behavior and tool selection.<n>We present TRUSTDESC, the first framework for preventing tool poisoning by automatically generating trusted tool descriptions from implementations.
  arXiv  Detail & Related papers  (2026-04-08T19:18:11Z)
• SkillJect: Automating Stealthy Skill-Based Prompt Injection for Coding Agents with Trace-Driven Closed-Loop Refinement [120.52289344734415]
  We propose an automated framework for stealthy prompt injection tailored to agent skills.<n>The framework forms a closed loop with three agents: an Attack Agent that synthesizes injection skills under explicit stealth constraints, a Code Agent that executes tasks using the injected skills and an Evaluate Agent that logs action traces.<n>Our method consistently achieves high attack success rates under realistic settings.
  arXiv  Detail & Related papers  (2026-02-15T16:09:48Z)
• MalTool: Malicious Tool Attacks on LLM Agents [52.01975462609959]
  MalTool is a coding-LLM-based framework that synthesizes tools exhibiting specified malicious behaviors.<n>We show that MalTool is highly effective even when coding LLMs are safety-aligned.
  arXiv  Detail & Related papers  (2026-02-12T17:27:43Z)
• IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents [33.775221377823925]
  Large language model (LLM) agents are widely deployed in real-world applications, where they leverage tools to retrieve and manipulate external data for complex tasks.<n>When interacting with untrusted data sources, tool responses may contain injected instructions that covertly influence agent behaviors and lead to malicious outcomes.<n>We propose a novel defensive task execution paradigm, called IPIGuard, to prevent malicious tool invocations at the source.
  arXiv  Detail & Related papers  (2025-08-21T07:08:16Z)
• Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools [10.086284534400658]
  Large language model (LLM) agents have demonstrated remarkable capabilities in complex reasoning and decision-making by leveraging external tools.<n>We identify this as a new and stealthy threat surface that allows malicious tools to be preferentially selected by LLM agents.<n>We propose a black-box in-context learning framework that generates highly attractive but syntactically and semantically valid tool metadata.
  arXiv  Detail & Related papers  (2025-08-04T06:38:59Z)
• Prompt Injection Attack to Tool Selection in LLM Agents [60.95349602772112]
  A popular approach follows a two-step process - emphretrieval and emphselection - to pick the most appropriate tool from a tool library for a given task.<n>In this work, we introduce textitToolHijacker, a novel prompt injection attack targeting tool selection in no-box scenarios.
  arXiv  Detail & Related papers  (2025-04-28T13:36:43Z)
• MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents [60.30753230776882]
  LLM agents are vulnerable to indirect prompt injection (IPI) attacks, where malicious tasks embedded in tool-retrieved information can redirect the agent to take unauthorized actions.<n>We present MELON, a novel IPI defense that detects attacks by re-executing the agent's trajectory with a masked user prompt modified through a masking function.
  arXiv  Detail & Related papers  (2025-02-07T18:57:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.