Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming

• URL: http://arxiv.org/abs/2606.31227v1
• Date: Tue, 30 Jun 2026 07:06:56 GMT
• Title: Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming
• Abstract summary: We present AI-Infra-Guard, an open-source framework that organizes AI red teaming around a single observation.<n>We release AI-Infra-Guard as open source so that emphlayer-paradigm matching can serve as a practical foundation for agent security.
• Score: 15.069910056925492
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The fast growth of open-source AI infrastructure, from model serving engines and agent platforms to the Model Context Protocol (MCP) ecosystem and the language models themselves, has outpaced the security tooling available to defend it. We present AI-Infra-Guard, an open-source framework that organizes AI red teaming around a single observation: the attack surface of an AI agent is stratified across layers (infrastructure, protocol/tool, agent behavior, and model), and no single detection paradigm fits all of them. The framework therefore matches a paradigm to each layer, from deterministic rule matching over 75+ AI components and 1{,}400+ vulnerability rules, through LLM-driven agentic auditing of MCP servers and agent-skill packages and multi-turn black-box agent red teaming, to a jailbreak harness with 26+ attack operators over sixteen datasets. To our knowledge it is the only open-source framework to span all of these, including supply-chain auditing of the agent skills that increasingly extend AI agents. We release AI-Infra-Guard as open source so that \emph{layer-paradigm matching} can serve as a practical foundation for agent security and a shared base for the community to build on.

Related papers

• Distributed General-Purpose Agent Networks: Architecture, Key Mechanisms, and Prototypes [6.912160696802991]
  This paper studies distributed general-purpose agent networks.<n>Unlike traditional P2P networks, agent networks must propagate semantic declarations about intentions, capabilities, states, and cooperation constraints.<n>We present a technical route, including bodyless gossip with sequential logs, BAID-based identity binding with MG-EigenTrust reputation, and a Stackelberg-style mechanism-generation loop driven by semantic attribution feedback.
  arXiv  Detail & Related papers  (2026-06-15T23:57:13Z)
• DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents [121.77550256034]
  DecodingTrust-Agent Platform (DTap) is a controllable and interactive red-teaming platform for AI agents.<n>DTap-Red is the first autonomous red-teaming agent that explores diverse injection vectors and autonomously discovers effective attack strategies.<n>Through DTap, we conduct large-scale evaluations of popular AI agents built on various backbone models.
  arXiv  Detail & Related papers  (2026-05-06T11:59:48Z)
• When Agents Handle Secrets: A Survey of Confidential Computing for Agentic AI [6.1511441479993225]
  Agentic AI systems that plan, invoke tools, maintain persistent memory, and delegate tasks to peer agents introduce a threat surface that differs materially from standalone model inference.<n> Confidential computing offers a hardware-rooted alternative: Trusted Execution Environments (TEEs) isolate agent code and data from privileged system software.<n>This survey synthesizes the design space in four parts: (i) a unified taxonomy of six TEE platforms covering deployment roles and performance tradeoffs; (ii) an agent-centric threat model spanning perception, planning, memory, action, and coordination layers mapped to nine security goals; (iii) a comparative survey of CC
  arXiv  Detail & Related papers  (2026-05-04T23:09:16Z)
• OMNI-LEAK: Orchestrator Multi-Agent Network Induced Data Leakage [59.3826294523924]
  We investigate the security vulnerabilities of a popular multi-agent pattern known as the orchestrator setup.<n>We report the susceptibility of frontier models to different categories of attacks, finding that both reasoning and non-reasoning models are vulnerable.
  arXiv  Detail & Related papers  (2026-02-13T21:32:32Z)
• Using the NANDA Index Architecture in Practice: An Enterprise Perspective [9.707223291705601]
  The proliferation of autonomous AI agents represents a paradigmatic shift from traditional web architectures toward collaborative intelligent systems.<n>This paper presents a comprehensive framework addressing the fundamental infrastructure requirements for secure, trustworthy, and interoperable AI agent ecosystems.
  arXiv  Detail & Related papers  (2025-08-05T05:27:27Z)
• Towards Unifying Quantitative Security Benchmarking for Multi Agent Systems [0.0]
  Evolving AI systems increasingly deploy multi-agent architectures where autonomous agents collaborate, share information, and delegate tasks through developing protocols.<n>One such risk is a cascading risk: a breach in one agent can cascade through the system, compromising others by exploiting inter-agent trust.<n>In an ACI attack, a malicious input or tool exploit injected at one agent leads to cascading compromises and amplified downstream effects across agents that trust its outputs.
  arXiv  Detail & Related papers  (2025-07-23T13:51:28Z)
• Kaleidoscopic Teaming in Multi Agent Simulations [75.47388708240042]
  We argue that existing red teaming or safety evaluation frameworks fall short in evaluating safety risks in complex behaviors, thought processes and actions taken by agents.<n>We introduce new in-context optimization techniques that can be used in our kaleidoscopic teaming framework to generate better scenarios for safety analysis.<n>We present appropriate metrics that can be used along with our framework to measure safety of agents.
  arXiv  Detail & Related papers  (2025-06-20T23:37:17Z)
• TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems [8.683314804719506]
  This review presents a structured analysis of Trust, Risk, and Security Management (TRiSM) in the context of Agentic Multi-Agent Systems (AMAS)<n>We begin by examining the conceptual foundations of Agentic AI and highlight its architectural distinctions from traditional AI agents.<n>We then adapt and extend the AI TRiSM framework for Agentic AI, structured around key pillars: textit Explainability, ModelOps, Security, Privacy and textittheir lifecycle governance<n>A risk taxonomy is proposed to capture the unique threats and vulnerabilities of Agentic AI, ranging from coordination failures to
  arXiv  Detail & Related papers  (2025-06-04T16:26:11Z)
• RedTeamLLM: an Agentic AI framework for offensive security [0.0]
  We propose and evaluate RedTeamLLM, an integrated architecture with a comprehensive security model for automatization of pentest tasks.<n>RedTeamLLM follows three key steps: summarizing, reasoning and act, which embed its operational capacity.<n> Evaluation is performed through the automated resolution of a range of entry-level, but not trivial, CTF challenges.
  arXiv  Detail & Related papers  (2025-05-11T09:19:10Z)
• AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents [54.29555239363013]
  We propose a generic black-box fuzzing framework, AgentVigil, to automatically discover and exploit indirect prompt injection vulnerabilities.<n>We evaluate AgentVigil on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o.<n>We apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.
  arXiv  Detail & Related papers  (2025-05-09T07:40:17Z)
• DoomArena: A framework for Testing AI Agents Against Evolving Security Threats [81.73540246946015]
  We present DoomArena, a security evaluation framework for AI agents.<n>It is a plug-in framework and integrates easily into realistic agentic frameworks.<n>It is modular and decouples the development of attacks from details of the environment in which the agent is deployed.
  arXiv  Detail & Related papers  (2025-04-18T20:36:10Z)
• Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence [79.5316642687565]
  Existing multi-agent frameworks often struggle with integrating diverse capable third-party agents. We propose the Internet of Agents (IoA), a novel framework that addresses these limitations. IoA introduces an agent integration protocol, an instant-messaging-like architecture design, and dynamic mechanisms for agent teaming and conversation flow control.
  arXiv  Detail & Related papers  (2024-07-09T17:33:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.