Addressing Corpus Knowledge Poisoning Attacks on RAG Using Sparse Attention

• URL: http://arxiv.org/abs/2602.04711v2
• Date: Thu, 05 Feb 2026 07:59:00 GMT
• Title: Addressing Corpus Knowledge Poisoning Attacks on RAG Using Sparse Attention
• Abstract summary: We introduce a novel defense approach for RAG: Sparse Document Attention RAG (SDAG)<n> SDAG is a block-sparse attention mechanism that disallows cross-attention between retrieved documents.<n>We show that our SDAG method substantially outperforms the standard causal attention mechanism in terms of attack success rate.
• Score: 10.71610316323851
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval Augmented Generation (RAG) is a highly effective paradigm for keeping LLM-based responses up-to-date and reducing the likelihood of hallucinations. Yet, RAG was recently shown to be quite vulnerable to corpus knowledge poisoning: an attacker injects misleading documents to the corpus to steer an LLM's output to an undesired response. We argue that the standard causal attention mechanism in LLMs enables harmful cross-document interactions, specifically in cases of attacks. Accordingly, we introduce a novel defense approach for RAG: Sparse Document Attention RAG (SDAG). This is a block-sparse attention mechanism that disallows cross-attention between retrieved documents. SDAG requires a minimal inference-time change to the attention mask; furthermore, no fine-tuning or additional architectural changes are needed. We present an empirical evaluation of LLM-based question answering (QA) with a variety of attack strategies on RAG. We show that our SDAG method substantially outperforms the standard causal attention mechanism in terms of attack success rate. We further demonstrate the clear merits of integrating SDAG with state-of-the-art RAG defense methods. Specifically, the integration results in performance that is statistically significantly better than the state-of-the-art.

Related papers

• InceptionRAG: Stealthy Poisoning Attack Against Retrieval-Augmented Generation [23.65420330011369]
  We introduce InceptionRAG, a stealthy attack mechanism that subverts the standard attack paradigm.<n>InceptionRAG achieves an attack success rate exceeding 80% even under rigorous adversarial constraints.<n>Our findings expose a concerning paradox: the stronger reasoning capabilities of LLMs increase their vulnerability to reasoning-based poisoning attacks.
  arXiv  Detail & Related papers  (2026-09-15T08:22:10Z)
• AdversarialCoT: Single-Document Retrieval Poisoning for LLM Reasoning [115.3243260783674]
  Retrieval-augmented generation (RAG) enhances large language model (LLM) reasoning by retrieving external documents.<n>We study knowledge-base poisoning attacks in RAG, where an attacker injects malicious content into the retrieval corpus.<n>We propose AdversarialCoT, a query-specific attack that poisons only a single document in the corpus.
  arXiv  Detail & Related papers  (2026-04-14T02:10:23Z)
• RAGPart & RAGMask: Retrieval-Stage Defenses Against Corpus Poisoning in Retrieval-Augmented Generation [43.85099769473328]
  Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm to enhance large language models.<n>Recent studies have exposed a critical vulnerability in RAG pipelines corpus poisoning where adversaries inject malicious documents into the retrieval corpus to manipulate model outputs.<n>We propose two complementary retrieval-stage defenses: RAGPart and RAGMask.
  arXiv  Detail & Related papers  (2025-12-30T14:43:57Z)
• RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement Learning [23.957879891712306]
  We propose an end-to-end attack pipeline that treats the target RAG system as a black box.<n>We demonstrate that this method can effectively execute poisoning attacks against most complex RAG systems.
  arXiv  Detail & Related papers  (2025-10-11T04:23:20Z)
• The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems [101.68501850486179]
  We explore adversarial attacks against retrieval-augmented generation (RAG) systems to identify their vulnerabilities.<n>This task aims to find imperceptible perturbations that retrieve a target document, originally excluded from the initial top-$k$ candidate set.<n>We propose ReGENT, a reinforcement learning-based framework that tracks interactions between the attacker and the target RAG.
  arXiv  Detail & Related papers  (2025-05-24T08:19:25Z)
• Benchmarking Poisoning Attacks against Retrieval-Augmented Generation [12.573766276297441]
  Retrieval-Augmented Generation (RAG) has proven effective in mitigating hallucinations in large language models by incorporating external knowledge during inference.<n>We propose the first comprehensive benchmark framework for evaluating poisoning attacks on RAG.
  arXiv  Detail & Related papers  (2025-05-24T06:17:59Z)
• Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization [58.390885294401066]
  Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs)<n>RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions.<n>We propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA)<n>We introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations.
  arXiv  Detail & Related papers  (2025-04-21T04:56:47Z)
• Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generation [71.32665836294103]
  Multimodal retrieval-augmented generation (RAG) enhances the visual reasoning capability of vision-language models (VLMs)<n>In this work, we introduce textitPoisoned-MRAG, the first knowledge poisoning attack on multimodal RAG systems.
  arXiv  Detail & Related papers  (2025-03-08T15:46:38Z)
• MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks [104.50239783909063]
  Multimodal large language models with Retrieval Augmented Generation (RAG) have significantly advanced tasks such as multimodal question answering.<n>This reliance on external knowledge poses a critical yet underexplored safety risk: knowledge poisoning attacks.<n>We propose MM-PoisonRAG, the first framework to systematically design knowledge poisoning in multimodal RAG.
  arXiv  Detail & Related papers  (2025-02-25T04:23:59Z)
• FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Models [22.35026334463735]
  We propose FlippedRAG, a transfer-based adversarial attack against black-box RAG systems.<n>FlippedRAG achieves on average a 50% directional shift in the opinion of RAG-generated responses.<n>These results highlight an urgent need for developing innovative defensive solutions to ensure the security and trustworthiness of RAG systems.
  arXiv  Detail & Related papers  (2025-01-06T12:24:57Z)
• Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworks [12.061098193438022]
  Retrieval Augmented Generation (RAG) is a technique commonly used to equip models with out of distribution knowledge. This paper investigates the security of RAG systems against end-to-end indirect prompt manipulations.
  arXiv  Detail & Related papers  (2024-08-09T12:26:05Z)
• Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting [68.90949377014742]
  Speculative RAG is a framework that leverages a larger generalist LM to efficiently verify multiple RAG drafts produced in parallel by a smaller, distilled specialist LM.<n>Our method accelerates RAG by delegating drafting to the smaller specialist LM, with the larger generalist LM performing a single verification pass over the drafts.<n>It notably enhances accuracy by up to 12.97% while reducing latency by 50.83% compared to conventional RAG systems on PubHealth.
  arXiv  Detail & Related papers  (2024-07-11T06:50:19Z)
• A Theory for Token-Level Harmonization in Retrieval-Augmented Generation [76.75124161306795]
  Retrieval-augmented generation (RAG) utilizes retrieved texts to enhance large language models (LLMs)<n>This paper provides a theory to explain and trade off the benefit and detriment in RAG.<n>Based on our theory, we propose a practical novel method, Tok-RAG, which achieves collaborative generation between the pure LLM and RAG.
  arXiv  Detail & Related papers  (2024-06-03T02:56:14Z)
• Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations [9.209974698634175]
  Retrieval-Augmented Generation (RAG) is a promising solution for addressing the limitations of Large Language Models (LLMs) In this work, we investigate two underexplored aspects when assessing the robustness of RAG. We introduce a novel attack method, the Genetic Attack on RAG (textitGARAG), which targets these aspects.
  arXiv  Detail & Related papers  (2024-04-22T07:49:36Z)
• PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models [45.409248316497674]
  Large language models (LLMs) have achieved remarkable success due to their exceptional generative capabilities. Retrieval-Augmented Generation (RAG) is a state-of-the-art technique to mitigate these limitations. We find that the knowledge database in a RAG system introduces a new and practical attack surface. Based on this attack surface, we propose PoisonedRAG, the first knowledge corruption attack to RAG.
  arXiv  Detail & Related papers  (2024-02-12T18:28:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.