Cordon-MAS: Defending RAG against Knowledge Poisoning via Information-Flow Control

• URL: http://arxiv.org/abs/2605.26754v1
• Date: Tue, 26 May 2026 09:27:19 GMT
• Title: Cordon-MAS: Defending RAG against Knowledge Poisoning via Information-Flow Control
• Abstract summary: Existing defenses assume that detecting poisoned evidence prevents harm.<n>We show this assumption is incorrect: models exhibit a monitoring-control gap.<n>We introduce the Cordon Principle -- no agent capable of final synthesis may access untrusted natural-language evidence.
• Score: 22.814552066611597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-augmented generation (RAG) increasingly underpins high-stakes applications, yet remains vulnerable to Confundo-style poisoning where adversarially optimized documents manipulate generated outputs. Existing defenses assume that detecting poisoned evidence prevents harm. We show this assumption is incorrect: models exhibit a monitoring-control gap -- they can detect contradictions in retrieved evidence yet still act on poisoned claims. We introduce the Cordon Principle -- no agent capable of final synthesis may access untrusted natural-language evidence -- and realize it through CORDON-MAS, a compartmentalized framework that enforces this principle architecturally by separating evidence extraction, cross-source audit, and answer synthesis into agents with asymmetric memory privileges. Across five BEIR datasets, CORDON-MAS reduces attack success rate by 92.4\% relative to undefended RAG. This reframes RAG poisoning from a detection problem to an information-flow control problem.

Related papers

• Towards Safer RAG: Only Agents Capable of System 2 Thinking may Access Untrusted Documents [0.0]
  Retrieval-Augmented Generation (RAG) has significantly enhanced the performance of large language models.<n>RAG is vulnerable to knowledge-poisoning attacks in which misinformation in retrieved documents can influence the model's final outputs.<n>We propose a refined security principle: only agents capable of deliberative System 2 reasoning may access untrusted documents.
  arXiv  Detail & Related papers  (2026-08-17T21:37:20Z)
• Combating Knowledge Corruption in Agent Systems: A Byzantine-Tolerant Secure Collaborative RAG Framework [61.79960727298522]
  We propose SecureCollaRAG, a Byzantine-tolerant collaborative RAG framework.<n>Our approach enables agent system to securely verify document provenance through dynamic GNN-based credibility scoring.<n>Through extensive evaluations and formal analysis, we demonstrate that SecureCollaRAG maintains robustness against attackers under non-IID data distributions.
  arXiv  Detail & Related papers  (2026-08-05T02:13:37Z)
• Evading Chain-of-Thought Monitoring Through Model Poisoning [1.6951239729573633]
  Chain-of-thought (CoT) monitoring is an increasingly important component of AI safety stacks.<n>This work studies the limits of CoT monitoring through the lens of model poisoning.<n> backdoors can be implanted into reasoning models to elicit an attacker-chosen behavior while their CoT traces appear entirely benign.
  arXiv  Detail & Related papers  (2026-08-03T19:32:13Z)
• Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs [20.59321114618083]
  We show that single-turn diagnostics systematically overestimate RAG safety, that contradiction is uncorrelated with safe resolution, and that no universal prompt fix exists.<n>The gap between what models recognize and what they do must be measured and closed before retrieval-augmented systems can be trusted in high-stakes settings.
  arXiv  Detail & Related papers  (2026-05-26T15:18:43Z)
• Gray-Box Poisoning of Continuous Malware Ingestion Pipelines [0.0]
  This work investigates a realistic gray-box poisoning threat model targeting modern malware detection pipelines.<n>We generate problem-space adversarial binaries through functionality-preserving manipulations.<n>We evaluate the impact of these poisoned samples when ingested into a defender's training set for a LightGBM malware detection model.
  arXiv  Detail & Related papers  (2026-05-06T09:49:23Z)
• AdversarialCoT: Single-Document Retrieval Poisoning for LLM Reasoning [115.3243260783674]
  Retrieval-augmented generation (RAG) enhances large language model (LLM) reasoning by retrieving external documents.<n>We study knowledge-base poisoning attacks in RAG, where an attacker injects malicious content into the retrieval corpus.<n>We propose AdversarialCoT, a query-specific attack that poisons only a single document in the corpus.
  arXiv  Detail & Related papers  (2026-04-14T02:10:23Z)
• Confundo: Learning to Generate Robust Poison for Practical RAG Systems [19.77771071590713]
  Confundo is a learning-to-poison framework that fine-tunes a large language model as a poison generator to achieve high effectiveness, robustness, and stealthiness.<n>We show how Confundo consistently outperforms a wide range of purpose-built attacks across datasets and RAG configurations.<n>We also present a defensive use case that protects web content from unauthorized incorporation into RAG systems via scraping.
  arXiv  Detail & Related papers  (2026-02-06T11:19:49Z)
• ADMIT: Few-shot Knowledge Poisoning Attacks on RAG-based Fact Checking [42.283307608442946]
  Knowledge poisoning poses a critical threat to Retrieval-Augmented Generation (RAG) systems.<n>textbfADMIT (textbfADversarial textbfMulti-textbfInjection textbfTechnique) is a semantically aligned poisoning attack that flips fact-checking decisions.
  arXiv  Detail & Related papers  (2025-10-11T14:50:40Z)
• Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain [82.98626829232899]
  Fine-tuning AI agents on data from their own interactions introduces a critical security vulnerability within the AI supply chain.<n>We show that adversaries can easily poison the data collection pipeline to embed hard-to-detect backdoors.
  arXiv  Detail & Related papers  (2025-10-03T12:47:21Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks [104.50239783909063]
  Multimodal large language models with Retrieval Augmented Generation (RAG) have significantly advanced tasks such as multimodal question answering.<n>This reliance on external knowledge poses a critical yet underexplored safety risk: knowledge poisoning attacks.<n>We propose MM-PoisonRAG, the first framework to systematically design knowledge poisoning in multimodal RAG.
  arXiv  Detail & Related papers  (2025-02-25T04:23:59Z)
• PoisonCatcher: Revealing and Identifying LDP Poisoning Attacks in IIoT [13.68394346583211]
  Local Differential Privacy (LDP) is widely adopted in the Industrial Internet of Things (IIoT) due to its lightweight, decentralized, and scalable.<n>This work proposes a LDP poisoning defense for IIoT in the resource-rich aggregator.
  arXiv  Detail & Related papers  (2024-12-20T09:26:50Z)
• Towards A Proactive ML Approach for Detecting Backdoor Poison Samples [38.21287048132065]
  Adversaries can embed backdoors in deep learning models by introducing backdoor poison samples into training datasets. In this work, we investigate how to detect such poison samples to mitigate the threat of backdoor attacks.
  arXiv  Detail & Related papers  (2022-05-26T20:44:15Z)
• No Need to Know Physics: Resilience of Process-based Model-free Anomaly Detection for Industrial Control Systems [95.54151664013011]
  We present a novel framework to generate adversarial spoofing signals that violate physical properties of the system. We analyze four anomaly detectors published at top security conferences.
  arXiv  Detail & Related papers  (2020-12-07T11:02:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.