SoK: The Attack Surface of Agentic AI -- Tools, and Autonomy

• URL: http://arxiv.org/abs/2603.22928v1
• Date: Tue, 24 Mar 2026 08:21:51 GMT
• Title: SoK: The Attack Surface of Agentic AI -- Tools, and Autonomy
• Abstract summary: Recent AI systems combine large language models with tools, external knowledge via retrieval-augmented generation (RAG)<n>This agentic AI paradigm greatly expands capabilities - but also vastly enlarges the attack surface.<n>We develop a comprehensive taxonomy of attacks spanning prompt-level injections, knowledge-base poisoning, tool/plug-in exploits, and multi-agent emergent threats.
• Score: 3.340255811686752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent AI systems combine large language models with tools, external knowledge via retrieval-augmented generation (RAG), and even autonomous multi-agent decision loops. This agentic AI paradigm greatly expands capabilities - but also vastly enlarges the attack surface. In this systematization, we map out the trust boundaries and security risks of agentic LLM-based systems. We develop a comprehensive taxonomy of attacks spanning prompt-level injections, knowledge-base poisoning, tool/plug-in exploits, and multi-agent emergent threats. Through a detailed literature review, we synthesize evidence from 2023-2025, including more than 20 peer-reviewed and archival studies, industry reports, and standards. We find that agentic systems introduce new vectors for indirect prompt injection, code execution exploits, RAG index poisoning, and cross-agent manipulation that go beyond traditional AI threats. We define attacker models and threat scenarios, and propose metrics (e.g., Unsafe Action Rate, Privilege Escalation Distance) to evaluate security posture. Our survey examines defenses such as input sanitization, retrieval filters, sandboxes, access control, and "AI guardrails," assessing their effectiveness and pointing out the areas where protection is still lacking. To assist practitioners, we outline defensive controls and provide a phased security checklist for deploying agentic AI (covering design-time hardening, runtime monitoring, and incident response). Finally, we outline open research challenges in secure autonomous AI (robust tool APIs, verifiable agent behavior, supply-chain safeguards) and discuss ethical and responsible disclosure practices. We systematize recent findings to help researchers and engineers understand and mitigate security risks in agentic AI.

Related papers

• Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectives [0.0]
  LLM-powered autonomous agents are transforming the penetration testing space with dynamic, multi-step offensive security.<n>But the ability to have persistent memory, to take actions in the real world, and to do long-horizon reasoning raises qualitatively different security concerns.<n>Existing guardrail mechanisms for conversational AI may not be sufficient to secure autonomous AI pentesting agents accordingly.
  arXiv  Detail & Related papers  (2026-09-15T06:19:50Z)
• DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents [121.77550256034]
  DecodingTrust-Agent Platform (DTap) is a controllable and interactive red-teaming platform for AI agents.<n>DTap-Red is the first autonomous red-teaming agent that explores diverse injection vectors and autonomously discovers effective attack strategies.<n>Through DTap, we conduct large-scale evaluations of popular AI agents built on various backbone models.
  arXiv  Detail & Related papers  (2026-05-06T11:59:48Z)
• Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses [168.50301366360344]
  Embodied Artificial Intelligence (Embodied AI) integrates perception, cognition, planning, and interaction into agents that operate in safety-critical environments.<n>Unlike digital AI systems, embodied agents must act under uncertain sensing, incomplete knowledge, and dynamic human-robot interactions.<n>This survey provides a comprehensive review of safety research in embodied AI, examining attacks and defenses across the full embodied pipeline.
  arXiv  Detail & Related papers  (2026-03-28T13:21:44Z)
• Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClaw [42.9467373594646]
  This report presents a comprehensive security analysis of the OpenClaw ecosystem.<n>We highlight critical vulnerabilities such as prompt injection-driven Remote Code Execution (RCE), sequential tool attack chains, context amnesia, and supply chain contamination.<n>To address these systemic architectural flaws, we introduce the Full-Lifecycle Agent Security Architecture (FASA)<n>This theoretical defense blueprint advocates for zero-trust agentic execution, dynamic intent verification, and cross-layer reasoning-action correlation.
  arXiv  Detail & Related papers  (2026-03-13T04:33:05Z)
• From Secure Agentic AI to Secure Agentic Web: Challenges, Threats, and Future Directions [20.73038673205127]
  We provide a transition-oriented view from Secure Agentic AI to a Secure Agentic Web.<n>We first summarize a component-aligned threat taxonomy covering prompt abuse, environment injection, memory attacks, toolchain abuse, model tampering, and agent network attacks.<n>We then review defense strategies, including prompt hardening, safety-aware decoding, privilege control for tools and APIs, runtime monitoring, continuous red-teaming, and protocol-level security mechanisms.
  arXiv  Detail & Related papers  (2026-03-02T07:44:18Z)
• Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report v1.5 [61.787178868669265]
  This technical report presents an updated and granular assessment of five critical dimensions: cyber offense, persuasion and manipulation, strategic deception, uncontrolled AI R&D, and self-replication.<n>This work reflects our current understanding of AI frontier risks and urges collective action to mitigate these challenges.
  arXiv  Detail & Related papers  (2026-02-16T04:30:06Z)
• OMNI-LEAK: Orchestrator Multi-Agent Network Induced Data Leakage [59.3826294523924]
  We investigate the security vulnerabilities of a popular multi-agent pattern known as the orchestrator setup.<n>We report the susceptibility of frontier models to different categories of attacks, finding that both reasoning and non-reasoning models are vulnerable.
  arXiv  Detail & Related papers  (2026-02-13T21:32:32Z)
• Agent2Agent Threats in Safety-Critical LLM Assistants: A Human-Centric Taxonomy [4.058281338403478]
  We propose a threat modeling framework called AgentHeLLM that separates asset identification from attack path analysis.<n>We introduce a human-centric asset taxonomy derived from harm-oriented "victim modeling" and inspired by the Universal Declaration of Human Rights.<n>We demonstrate the framework's practical applicability through an open-source attack path suggestion tool AgentHeLLM Attack Path Generator.
  arXiv  Detail & Related papers  (2026-02-05T16:53:41Z)
• Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs [65.6660735371212]
  We present textbftextscJustAsk, a framework that autonomously discovers effective extraction strategies through interaction alone.<n>It formulates extraction as an online exploration problem, using Upper Confidence Bound--based strategy selection and a hierarchical skill space spanning atomic probes and high-level orchestration.<n>Our results expose system prompts as a critical yet largely unprotected attack surface in modern agent systems.
  arXiv  Detail & Related papers  (2026-01-29T03:53:25Z)
• The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines [0.0]
  Cybersecurity has become one of the earliest adopters of agentic AI.<n>This survey presents a five-generation taxonomy of agentic AI in cybersecurity.
  arXiv  Detail & Related papers  (2025-12-07T05:10:16Z)
• A Systematic Survey of Model Extraction Attacks and Defenses: State-of-the-Art and Perspectives [65.3369988566853]
  Recent studies have demonstrated that adversaries can replicate a target model's functionality.<n>Model Extraction Attacks pose threats to intellectual property, privacy, and system security.<n>We propose a novel taxonomy that classifies MEAs according to attack mechanisms, defense approaches, and computing environments.
  arXiv  Detail & Related papers  (2025-08-20T19:49:59Z)
• Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition [101.86739402748995]
  We run the largest public red-teaming competition to date, targeting 22 frontier AI agents across 44 realistic deployment scenarios.<n>We build the Agent Red Teaming benchmark and evaluate it across 19 state-of-the-art models.<n>Our findings highlight critical and persistent vulnerabilities in today's AI agents.
  arXiv  Detail & Related papers  (2025-07-28T05:13:04Z)
• Artificial Intelligence as the New Hacker: Developing Agents for Offensive Security [0.0]
  This paper explores the integration of Artificial Intelligence (AI) into offensive cybersecurity. It develops an autonomous AI agent, ReaperAI, designed to simulate and execute cyberattacks. ReaperAI demonstrates the potential to identify, exploit, and analyze security vulnerabilities autonomously.
  arXiv  Detail & Related papers  (2024-05-09T18:15:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.