Securing LLM Agents Need Intent-to-Execution Integrity

• URL: http://arxiv.org/abs/2605.16976v1
• Date: Sat, 16 May 2026 12:53:31 GMT
• Title: Securing LLM Agents Need Intent-to-Execution Integrity
• Abstract summary: We argue that securing LLM agents requires defining an end-to-end correctness property that specifies when an agent's execution faithfully reflects the user's intent.<n>We observe that LLM agents are structurally analogous to compilers, where security violations correspond to mis-executions that do not preserve user intent.<n>We derive four integrity properties that must hold simultaneously: emphTool Integrity, emphInstruction Integrity, emphJudgment Integrity, and emphData Flow Integrity.
• Score: 49.490963596514185
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This position paper argues that securing LLM agents requires first defining an end-to-end correctness property that specifies when an agent's execution faithfully reflects the user's intent. Modern LLM agents operate over an \emph{intent-to-execution pipeline}, where natural-language instructions are translated into concrete system operations such as tool calls, API requests, and code execution. While recent defenses have made progress in constraining how agents construct tool calls, most existing formulations implicitly assume that tools are trusted. The emergence of systems such as OpenClaw, with open ecosystems of third-party skills and direct access to user environments, breaks this assumption and exposes new failure modes, including malicious or over-privileged components in the execution pipeline. Despite rapid progress in defense mechanisms, there is no adequate correctness property that defines what ``secure'' means for LLM agents, nor a principled way to evaluate the coverage of existing defenses. We observe that LLM agents are structurally analogous to compilers, where security violations correspond to mis-executions that do not preserve user intent. Drawing on this analogy, we identify two fundamental problem sources -- untrusted data ingestion and untrusted tool execution -- and derive four integrity properties that must hold simultaneously: \emph{Tool Integrity}, \emph{Instruction Integrity}, \emph{Judgment Integrity}, and \emph{Data Flow Integrity}. We call their conjunction \emph{intent-to-execution integrity}. Analyzing existing agentic defenses against these properties reveals that current systems provide only partial and non-compositional coverage, leaving fundamental gaps in securing modern LLM agents.

Related papers

• ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection [61.4148196085256]
  We present ROPE (Routed Origin Policy Enforcement), which is anchored in a structural notion of trust.<n>Our approach admits two provable guarantees: 1) at every step of a trajectory, no value whose only origin is attacker-writable content reaches an origin-guarded parameter, and 2) no rewording of an injection changes an admission decision.
  arXiv  Detail & Related papers  (2026-08-27T01:22:14Z)
• Formal Verification of Agentic Systems over Operational Data [59.98246281888422]
  Agentic systems driven by large language models (LLMs) are increasingly deployed in real-world where they act on persistent operational data.<n>Existing approaches do not provide such system-level guarantees.<n>We study here the verification of agentic systems comprising a single LLM and a tool orchestration harness over relational operational data.
  arXiv  Detail & Related papers  (2026-08-04T13:01:30Z)
• Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation [13.654207263751156]
  Large language model (LLM) agents are rapidly moving from conversational interfaces to software components that plan, invoke tools, maintain memory, and act on external environments.<n>This paper synthesizes 247 papers through a lifecycle-based, systems-oriented framework that models agent security around the interaction of information flow, delegated authority, and persistent state.<n>We find that prompt injection and tool-mediated control-flow hijacking still dominate the field, while persistent state corruption and multi-agent propagation are becoming central emerging concerns.
  arXiv  Detail & Related papers  (2026-06-09T12:01:07Z)
• PragLocker: Protecting Agent Intellectual Property in Untrusted Deployments via Non-Portable Prompts [18.735200590753525]
  LLM agents rely on prompts to implement task-specific capabilities based on foundation LLMs.<n>In untrusted deployments, adversaries can copy and reuse these prompts with other proprietary LLMs, causing economic losses.<n>We present PragLocker, a prompt protection scheme that satisfies these requirements.
  arXiv  Detail & Related papers  (2026-05-07T10:19:06Z)
• ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection [8.627671856044527]
  textscClawGuard enforces a user-confirmed rule set at every tool-call boundary.<n>textscClawGuard blocks all three injection pathways without model modification or infrastructure change.<n>This work establishes deterministic tool-call boundary enforcement as an effective defense mechanism for secure agentic AI systems.
  arXiv  Detail & Related papers  (2026-04-13T17:55:11Z)
• A Framework for Formalizing LLM Agent Security [46.86073273904095]
  Security in LLM agents is inherently contextual.<n>Existing definitions of security attacks against LLM agents often fail to capture this contextual nature.<n>We present a framework that systematizes existing attacks and defenses from the perspective of contextual security.
  arXiv  Detail & Related papers  (2026-03-19T21:01:49Z)
• You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents [9.719776777345364]
  High-privilege LLM agents that autonomously process external documentation are increasingly trusted to automate tasks.<n>These agents are granted terminal access, control, and outbound network connectivity with minimal security oversight.<n>We measure a fundamental vulnerability in this trust model, which we term the emphTrusted Executor Dilemma<n>This vulnerability is a structural consequence of the instruction-following design paradigm, not an implementation bug.
  arXiv  Detail & Related papers  (2026-03-12T12:35:46Z)
• CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents [60.98294016925157]
  AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior to steal credentials or cause financial loss.<n>We introduce Single-Shot Planning for CUAs, where a trusted planner generates a complete execution graph with conditional branches before any observation of potentially malicious content.<n>Although this architectural isolation successfully prevents instruction injections, we show that additional measures are needed to prevent Branch Steering attacks.
  arXiv  Detail & Related papers  (2026-01-14T23:06:35Z)
• Towards Verifiably Safe Tool Use for LLM Agents [53.55621104327779]
  Large language model (LLM)-based AI agents extend capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents.<n>LLMs may invoke unintended tool interactions and introduce risks, such as leaking sensitive data or overwriting critical records.<n>Current approaches to mitigate these risks, such as model-based safeguards, enhance agents' reliability but cannot guarantee system safety.
  arXiv  Detail & Related papers  (2026-01-12T21:31:38Z)
• Securing the Model Context Protocol: Defending LLMs Against Tool Poisoning and Adversarial Attacks [8.419049623790618]
  This work analyzes three classes of semantic attacks on MCP-integrated systems.<n>We introduce a layered security framework with three components: RSA-based manifest signing to enforce descriptor integrity, LLM-on-LLM semantic vetting to detect suspicious tool definitions, and lightweight guardrails that block anomalous tool behavior at runtime.<n>Our results show that the proposed framework reduces unsafe tool invocation rates without model fine-tuning or internal modification.
  arXiv  Detail & Related papers  (2025-12-06T20:07:58Z)
• LLM Agents Should Employ Security Principles [60.03651084139836]
  This paper argues that the well-established design principles in information security should be employed when deploying Large Language Model (LLM) agents at scale.<n>We introduce AgentSandbox, a conceptual framework embedding these security principles to provide safeguards throughout an agent's life-cycle.
  arXiv  Detail & Related papers  (2025-05-29T21:39:08Z)
• AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting [54.931241667414184]
  We propose textbfAdaptive textbfShield Prompting, which prepends inputs with defense prompts to defend MLLMs against structure-based jailbreak attacks. Our methods can consistently improve MLLMs' robustness against structure-based jailbreak attacks.
  arXiv  Detail & Related papers  (2024-03-14T15:57:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.