Semantic Chameleon: Corpus-Dependent Poisoning Attacks and Defenses in RAG Systems

• URL: http://arxiv.org/abs/2603.18034v1
• Date: Tue, 10 Mar 2026 23:15:13 GMT
• Title: Semantic Chameleon: Corpus-Dependent Poisoning Attacks and Defenses in RAG Systems
• Abstract summary: Retrieval-Augmented Generation (RAG) systems extend large language models with external knowledge sources.<n>In particular, adversaries can poison retrieval corpora so that malicious documents are preferentially retrieved at inference time.<n>We study gradient-guided corpus poisoning attacks against modern RAG pipelines and evaluate retrieval-layer defenses.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-Augmented Generation (RAG) systems extend large language models (LLMs) with external knowledge sources but introduce new attack surfaces through the retrieval pipeline. In particular, adversaries can poison retrieval corpora so that malicious documents are preferentially retrieved at inference time, enabling targeted manipulation of model outputs. We study gradient-guided corpus poisoning attacks against modern RAG pipelines and evaluate retrieval-layer defenses that require no modification to the underlying LLM. We implement dual-document poisoning attacks consisting of a sleeper document and a trigger document optimized using Greedy Coordinate Gradient (GCG). In a large-scale evaluation on the Security Stack Exchange corpus (67,941 documents) with 50 attack attempts, gradient-guided poisoning achieves a 38.0 percent co-retrieval rate under pure vector retrieval. We show that a simple architectural modification, hybrid retrieval combining BM25 and vector similarity, substantially mitigates this attack. Across all 50 attacks, hybrid retrieval reduces gradient-guided attack success from 38 percent to 0 percent without modifying the model or retraining the retriever. When attackers jointly optimize payloads for both sparse and dense retrieval signals, hybrid retrieval can be partially circumvented, achieving 20-44 percent success, but still significantly raises attack difficulty relative to vector-only retrieval. Evaluation across five LLM families (GPT-5.3, GPT-4o, Claude Sonnet 4.6, Llama 4, and GPT-4o-mini) shows attack success ranging from 46.7 percent to 93.3 percent. Cross-corpus evaluation on the FEVER Wikipedia dataset (25 attacks) yields 0 percent attack success across all retrieval configurations.

Related papers

• TRIS: A Tri-Layer Retrieval Integrity Sieve Against Knowledge Poisoning [6.613886785198943]
  Tri-Layer Sieve sanitizes retrieved evidence through cross-embedding-space clustering.<n>It exploits a key weakness of retrieval-stage poisoning: a single document must satisfy one embedding geometry, one internal Trigger-Payload structure, and one generation objective.
  arXiv  Detail & Related papers  (2026-08-31T23:21:27Z)
• RAGSieve: Self-Referenced Local Contrast for Knowledge-Poison Detection in Retrieval-Augmented Generation [0.0]
  Existing detectors depend on trusted references, specific attack artifacts, or global thresholds sensitive to corpus topology.<n>We present RAGSieve, a self-referenced detection framework that constructs its reference from the inspected system.<n>Joint deployment reduces attack success from 67.4% to 14.0% while retaining 41.3% F1 on unpoisoned retrieval.
  arXiv  Detail & Related papers  (2026-08-13T09:29:48Z)
• Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injection [0.0]
  We present RAG-IDS, a three-tier multi-agent intrusion detection framework with a retrieval-boundary defense.<n>Under prompt injection, multi-document retrieval limits label-flip success to 0.6-2.4%, compared with 35-55% for single-document retrieval.
  arXiv  Detail & Related papers  (2026-08-08T12:29:47Z)
• RAGPart & RAGMask: Retrieval-Stage Defenses Against Corpus Poisoning in Retrieval-Augmented Generation [43.85099769473328]
  Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm to enhance large language models.<n>Recent studies have exposed a critical vulnerability in RAG pipelines corpus poisoning where adversaries inject malicious documents into the retrieval corpus to manipulate model outputs.<n>We propose two complementary retrieval-stage defenses: RAGPart and RAGMask.
  arXiv  Detail & Related papers  (2025-12-30T14:43:57Z)
• Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systems [11.812488957698038]
  Large language models (LLMs) are reshaping numerous facets of our daily lives, leading widespread adoption as web-based services.<n>Retrieval-Augmented Generation (RAG) has emerged as a promising direction by generating responses grounded in external knowledge sources.<n>Recent studies demonstrate the vulnerability of RAG, such as knowledge corruption attacks by injecting misleading information.<n>In this work, we introduce RAGDefender, a resource-efficient defense mechanism against knowledge corruption.
  arXiv  Detail & Related papers  (2025-11-03T06:39:58Z)
• Joint-GCG: Unified Gradient-Based Poisoning Attacks on Retrieval-Augmented Generation Systems [11.300387488829035]
  Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by retrieving relevant documents from external corpora before generating responses.<n>Existing poisoning attack strategies treat the retrieval and generation stages as disjointed, limiting their effectiveness.<n>Joint-GCG is the first framework to unify gradient-based attacks across both retriever and generator models.
  arXiv  Detail & Related papers  (2025-06-06T15:12:06Z)
• CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models [15.349703228157479]
  Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by incorporating external knowledge.<n>Existing poisoning methods for RAG systems have limitations, such as poor generalization and lack of fluency in adversarial texts.<n>We propose CPA-RAG, a black-box adversarial framework that generates query-relevant texts capable of manipulating the retrieval process to induce target answers.
  arXiv  Detail & Related papers  (2025-05-26T11:48:32Z)
• Document Screenshot Retrievers are Vulnerable to Pixel Poisoning Attacks [72.4498910775871]
  Vision-language model (VLM)-based retrievers leverage document screenshots embedded as vectors to enable effective search and offer a simplified pipeline over traditional text-only methods.<n>In this study, we propose three pixel poisoning attack methods designed to compromise VLM-based retrievers.
  arXiv  Detail & Related papers  (2025-01-28T12:40:37Z)
• Corpus Poisoning via Approximate Greedy Gradient Descent [48.5847914481222]
  We propose Approximate Greedy Gradient Descent, a new attack on dense retrieval systems based on the widely used HotFlip method for generating adversarial passages. We show that our method achieves a high attack success rate on several datasets and using several retrievers, and can generalize to unseen queries and new domains.
  arXiv  Detail & Related papers  (2024-06-07T17:02:35Z)
• Improved Generation of Adversarial Examples Against Safety-aligned LLMs [72.38072942860309]
  Adversarial prompts generated using gradient-based methods exhibit outstanding performance in performing automatic jailbreak attacks against safety-aligned LLMs. In this paper, we explore a new perspective on this problem, suggesting that it can be alleviated by leveraging innovations inspired in transfer-based attacks. We show that 87% of the query-specific adversarial suffixes generated by the developed combination can induce Llama-2-7B-Chat to produce the output that exactly matches the target string on AdvBench.
  arXiv  Detail & Related papers  (2024-05-28T06:10:12Z)
• Poisoning Retrieval Corpora by Injecting Adversarial Passages [79.14287273842878]
  We propose a novel attack for dense retrieval systems in which a malicious user generates a small number of adversarial passages. When these adversarial passages are inserted into a large retrieval corpus, we show that this attack is highly effective in fooling these systems. We also benchmark and compare a range of state-of-the-art dense retrievers, both unsupervised and supervised.
  arXiv  Detail & Related papers  (2023-10-29T21:13:31Z)
• Guidance Through Surrogate: Towards a Generic Diagnostic Attack [101.36906370355435]
  We develop a guided mechanism to avoid local minima during attack optimization, leading to a novel attack dubbed Guided Projected Gradient Attack (G-PGA) Our modified attack does not require random restarts, large number of attack iterations or search for an optimal step-size. More than an effective attack, G-PGA can be used as a diagnostic tool to reveal elusive robustness due to gradient masking in adversarial defenses.
  arXiv  Detail & Related papers  (2022-12-30T18:45:23Z)
• Unified Detection of Digital and Physical Face Attacks [61.6674266994173]
  State-of-the-art defense mechanisms against face attacks achieve near perfect accuracies within one of three attack categories, namely adversarial, digital manipulation, or physical spoofs. We propose a unified attack detection framework, namely UniFAD, that can automatically cluster 25 coherent attack types belonging to the three categories.
  arXiv  Detail & Related papers  (2021-04-05T21:08:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.