Can It Reach the Generator? Investigating the Survival of Prompt-Injection Attacks in Realistic RAG Settings

• URL: http://arxiv.org/abs/2605.28017v2
• Date: Thu, 28 May 2026 01:52:56 GMT
• Title: Can It Reach the Generator? Investigating the Survival of Prompt-Injection Attacks in Realistic RAG Settings
• Abstract summary: We re-evaluate seven GEO attacks under a realistic three-stage pipeline.<n>We find that prior protocols substantially overstate attack effectiveness.<n>A lightweight prompt-injection guard finetuned on a small attack dataset already detects every attack.
• Score: 40.03039307576983
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent generative engine optimisation (GEO) research has shown that prompt-injection attacks can push a target product to the top of an LLM's recommendation list, with the strongest attacks reporting around $80\%$ success and raising serious security concerns about RAG-based recommendation. However, these results assume the attacked document is always fed directly to the generator, bypassing the retriever and reranker. This is unrealistic: in deployed RAG systems, the attack modifies the document content, which can in turn change whether the document is retrieved and reranked highly enough to reach the generator at all. In this paper, we re-evaluate seven GEO attacks under a realistic three-stage pipeline (retriever\,$\to$\,LLM reranker\,$\to$\,LLM generator). We find that prior protocols substantially overstate attack effectiveness: gradient-based and instruction override attacks largely collapse before reaching the generator, and only LLM-driven prompt injections remain effective end-to-end. Our analysis further reveals that current GEO attacks are easily detectable: a lightweight prompt-injection guard finetuned on a small attack dataset already detects every attack. Our code and data are available at https://github.com/ielab/geo_injection_rag_survival.

Related papers

• When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapse [34.49107596901488]
  RAGs are increasingly susceptible to poisoning attacks, in which adversarial documents are injected to manipulate generator outputs.<n>Previous methods rely on output-side signals such as perplexity and consistency checks to detect such attacks.<n>We propose textttD-SCAN, a lightweight detection framework that monitors attention dynamics to identify attacked generations.
  arXiv  Detail & Related papers  (2026-08-07T08:23:26Z)
• AutoDojo: Adaptive Black-Box Attacks Reveal the Limits of IPI Defenses and Task-Specification Effects in LLM Agents [57.34566159148893]
  Indirect prompt injection (IPI) is a major security threat to LLM-powered agents.<n>We develop AutoDojo, an adaptive extension of AgentDojo that optimize IPI against a given defense.
  arXiv  Detail & Related papers  (2026-06-13T02:09:08Z)
• RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement Learning [23.957879891712306]
  We propose an end-to-end attack pipeline that treats the target RAG system as a black box.<n>We demonstrate that this method can effectively execute poisoning attacks against most complex RAG systems.
  arXiv  Detail & Related papers  (2025-10-11T04:23:20Z)
• Disabling Self-Correction in Retrieval-Augmented Generation via Stealthy Retriever Poisoning [14.419943772894754]
  Retrieval-Augmented Generation (RAG) has become a standard approach for improving the reliability of large language models (LLMs)<n>This paper uncovers that such attacks could be mitigated by the strong textitself-correction ability (SCA) of modern LLMs.<n>We introduce textscDisarmRAG, a new poisoning paradigm that compromises the retriever itself to suppress the SCA and enforce attacker-chosen outputs.
  arXiv  Detail & Related papers  (2025-08-27T17:49:28Z)
• The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems [101.68501850486179]
  We explore adversarial attacks against retrieval-augmented generation (RAG) systems to identify their vulnerabilities.<n>This task aims to find imperceptible perturbations that retrieve a target document, originally excluded from the initial top-$k$ candidate set.<n>We propose ReGENT, a reinforcement learning-based framework that tracks interactions between the attacker and the target RAG.
  arXiv  Detail & Related papers  (2025-05-24T08:19:25Z)
• Rag and Roll: An End-to-End Evaluation of Indirect Prompt Manipulations in LLM-based Application Frameworks [12.061098193438022]
  Retrieval Augmented Generation (RAG) is a technique commonly used to equip models with out of distribution knowledge. This paper investigates the security of RAG systems against end-to-end indirect prompt manipulations.
  arXiv  Detail & Related papers  (2024-08-09T12:26:05Z)
• Phantom: General Backdoor Attacks on Retrieval Augmented Language Generation [44.74112207662136]
  Retrieval Augmented Generation (RAG) expands the capabilities of modern large language models (LLMs)<n>We propose a novel attack that allows an adversary to inject a single malicious document into a RAG system's knowledge base, and mount a backdoor poisoning attack.
  arXiv  Detail & Related papers  (2024-05-30T21:19:24Z)
• Backdoor Attack with Sparse and Invisible Trigger [57.41876708712008]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. backdoor attack is an emerging yet threatening training-phase threat. We propose a sparse and invisible backdoor attack (SIBA)
  arXiv  Detail & Related papers  (2023-05-11T10:05:57Z)
• RayS: A Ray Searching Method for Hard-label Adversarial Attack [99.72117609513589]
  We present the Ray Searching attack (RayS), which greatly improves the hard-label attack effectiveness as well as efficiency. RayS attack can also be used as a sanity check for possible "falsely robust" models.
  arXiv  Detail & Related papers  (2020-06-23T07:01:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.