Beyond Single-Agent Alignment: Preventing Context-Fragmented Violations in Multi-Agent Systems

• URL: http://arxiv.org/abs/2604.22879v1
• Date: Fri, 24 Apr 2026 03:08:52 GMT
• Title: Beyond Single-Agent Alignment: Preventing Context-Fragmented Violations in Multi-Agent Systems
• Abstract summary: We identify and formalize a novel security risk: Context-Fragmented Violations (CFVs)<n>CFVs are a class of policy breaches where individual agent actions appear locally safe and reasonable, yet collectively violate organizational policies.<n>We propose Distributed Sentinel, a distributed zero-trust enforcement architecture.
• Score: 10.660248467840821
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We identify and formalize a novel security risk: Context-Fragmented Violations (CFVs) - a class of policy breaches where individual agent actions appear locally safe and reasonable, yet collectively violate organizational policies because critical policy facts are siloed in different departments private contexts. Existing prompt-based alignment mechanisms and monolithic interceptors are poorly matched to violations that span contextual islands. We propose Distributed Sentinel, a distributed zero-trust enforcement architecture that introduces the Semantic Taint Token (STT) Protocol. Through lightweight sidecar proxies, our system propagates security state across organizational boundaries without exposing raw cross-domain data, enabling Counterfactual Graph Simulation for cross-domain policy verification. We construct PhantomEcosystem, a comprehensive benchmark comprising 9 categories of realistic cross-agent violation scenarios with adversarially balanced safe controls. On this benchmark, Distributed Sentinel achieves F1 = 0.95 with 106ms end-to-end latency (16ms verification + 90ms entity extraction on A100), compared to 0.85 F1 for prompt-based filtering and 0.65 for rule-based DLP. To empirically validate the need for external enforcement, we evaluate eight frontier LLMs in execution-oriented multi-agent workflows with per-agent domain world models. All models exhibit substantial violation rates (14-98%), with cross-domain data flows showing systematically higher violation rates than same-domain flows. These results indicate that self-avoidance is unreliable and that multi-agent security benefits from a centralized enforcement layer operating above individual agents.

Related papers

• EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents [74.54929751174289]
  EvoSafeHarness is a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain.<n>It reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost.<n>It also achieves the best score on Agent-SafetyBench for every victim.
  arXiv  Detail & Related papers  (2026-09-05T05:56:41Z)
• Privacy-Preserving Topology-Guided Safety for LLM-Based Multi-Agent Systems via Federated Graph Learning [67.04448659688579]
  FGLGuard cuts AgentDojo's ground-success rate by 43% at near-unguarded utility, zero API cost, and negligible capability loss.<n>Federation is not optional: off-the-shelf transfer collapses under distribution shift.<n>Live FGLGuard cuts AgentDojo's ground-success rate by 43% at near-unguarded utility, zero API cost, and negligible capability loss.
  arXiv  Detail & Related papers  (2026-09-02T07:57:12Z)
• Who Pays the Price? Stakeholder-Centric Prompt Injection Benchmarking for Real-world Web Agents [93.19140872946842]
  Web agents driven by large language models (LLMs) are increasingly deployed in real-world environments.<n>This makes them vulnerable to prompt-injection attacks, in which seemingly benign content embeds adversarial instructions that manipulate agent behaviour.<n>We introduce textbfsysname, a benchmark to systematically categorize and attribute harm in real-world web agent systems.
  arXiv  Detail & Related papers  (2026-06-11T14:12:43Z)
• Semantic Quorum Assurance: Collective Certification for Non-Deterministic AI Infrastructure [2.124730017640531]
  We introduce Quorum Semantic Assurance (SQA), a control-plane primitive for governing non-deterministic agentic infrastructure.<n>SQA represents proposals as declarative execution contracts bound to cryptographic evidence chains and routes them to a diverse panel of read-only, sandboxed validator agents.<n>On 500 infrastructure-inspired mutation scenarios, with safety results reported on held-out safe/unsafe trials excluding ambiguous scenarios, SQA reduces unsafe approval from 18.5% for single-agent validation to 0.3%.
  arXiv  Detail & Related papers  (2026-06-06T07:31:04Z)
• A New Framework for Cybersecurity Refusals in AI Agents [52.94784168139071]
  We present the first framework for establishing refusal boundaries in offensive security contexts.<n>We apply this framework to assess how current LLM-powered agents adhere to appropriate refusal boundaries across a range of web-based offensive security scenarios.
  arXiv  Detail & Related papers  (2026-05-31T15:39:39Z)
• DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents [121.77550256034]
  DecodingTrust-Agent Platform (DTap) is a controllable and interactive red-teaming platform for AI agents.<n>DTap-Red is the first autonomous red-teaming agent that explores diverse injection vectors and autonomously discovers effective attack strategies.<n>Through DTap, we conduct large-scale evaluations of popular AI agents built on various backbone models.
  arXiv  Detail & Related papers  (2026-05-06T11:59:48Z)
• GuardPhish: Securing Open-Source LLMs from Phishing Abuse [0.7009487789080343]
  Open-source Large Language Models (LLMs) in offline and enterprise environments have introduced a largely unexamined security risk.<n>We investigate this vulnerability through GuardPhish, a large scale multi-vector phishing prompt dataset.<n>Using a deterministic five model ensemble for labeling, we achieve near perfect inter model agreement.
  arXiv  Detail & Related papers  (2026-04-19T08:02:57Z)
• Parallax: Why AI Agents That Think Must Never Act [0.0]
  This paper introduces Parallax, a paradigm for safe autonomous AI execution grounded in four principles.<n>We present OpenParallax, an open-source reference implementation in Go, and evaluate it usingAssume-Compromise Evaluation.<n>Across 280 adversarial test cases in nine attack categories, Parallax blocks 98.9% of attacks with zero false positives under its default configuration, and 100% of attacks under its maximum-security configuration.
  arXiv  Detail & Related papers  (2026-04-14T17:20:48Z)
• LanG -- A Governance-Aware Agentic AI Platform for Unified Security Operations [2.1142550386295853]
  This paper presents the LLM-assisted network Governance (LanG), an open-source, governance-aware agentic AI platform for unified security operations.<n>The platform supports multi-tenant isolation, role-based access, and fully local deployment.<n>Finetuned anomaly and threat detectors achieve weighted F1 scores of 99.0% and 91.0%, respectively, in intrusion-detection benchmarks.
  arXiv  Detail & Related papers  (2026-04-07T05:22:25Z)
• Explainable Autonomous Cyber Defense using Adversarial Multi-Agent Reinforcement Learning [2.0391237204597368]
  We present the Causal Multi-Agent Decision Framework (C-MADF) for autonomous cyber defense.<n>C-MADF integrates causal modeling with adversarial dual-policy control.<n>On the real-world CICIoT2023 dataset, C-MADF reduces the false-positive rate from 11.2%, 9.7%, and 8.4% in three cutting-edge literature baselines to 1.8%.
  arXiv  Detail & Related papers  (2026-04-06T05:40:43Z)
• OMNI-LEAK: Orchestrator Multi-Agent Network Induced Data Leakage [59.3826294523924]
  We investigate the security vulnerabilities of a popular multi-agent pattern known as the orchestrator setup.<n>We report the susceptibility of frontier models to different categories of attacks, finding that both reasoning and non-reasoning models are vulnerable.
  arXiv  Detail & Related papers  (2026-02-13T21:32:32Z)
• Authenticated Workflows: A Systems Approach to Protecting Agentic AI [0.0]
  We introduce authenticated, the first complete trust layer for enterprise agentic AI.<n>We enforce intent (operations satisfy organizational policies) and integrity (operations are cryptographically authentic) at every boundary crossing.<n>This delivers deterministic security--operations either carry valid cryptographic proof or are rejected.
  arXiv  Detail & Related papers  (2026-02-11T03:04:50Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks [58.959622170433725]
  BlindGuard is an unsupervised defense method that learns without requiring any attack-specific labels or prior knowledge of malicious behaviors.<n>We show that BlindGuard effectively detects diverse attack types (i.e., prompt injection, memory poisoning, and tool attack) across multi-agent systems.
  arXiv  Detail & Related papers  (2025-08-11T16:04:47Z)
• Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition [101.86739402748995]
  We run the largest public red-teaming competition to date, targeting 22 frontier AI agents across 44 realistic deployment scenarios.<n>We build the Agent Red Teaming benchmark and evaluate it across 19 state-of-the-art models.<n>Our findings highlight critical and persistent vulnerabilities in today's AI agents.
  arXiv  Detail & Related papers  (2025-07-28T05:13:04Z)
• AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents [54.29555239363013]
  We propose a generic black-box fuzzing framework, AgentVigil, to automatically discover and exploit indirect prompt injection vulnerabilities.<n>We evaluate AgentVigil on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o.<n>We apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.
  arXiv  Detail & Related papers  (2025-05-09T07:40:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.