BudgetLeak: Membership Inference Attacks on RAG Systems via the Generation Budget Side Channel

• URL: http://arxiv.org/abs/2511.12043v1
• Date: Sat, 15 Nov 2025 05:25:04 GMT
• Title: BudgetLeak: Membership Inference Attacks on RAG Systems via the Generation Budget Side Channel
• Abstract summary: Membership inference attacks (MIAs) are a common technique to assess such risks.<n>We identify a previously unexplored side channel in RAG systems: the generation budget.<n>We propose BudgetLeak, a novel membership inference attack that probes responses under different budgets.
• Score: 21.74370009270649
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Retrieval-Augmented Generation (RAG) enhances large language models by integrating external knowledge, but reliance on proprietary or sensitive corpora poses various data risks, including privacy leakage and unauthorized data usage. Membership inference attacks (MIAs) are a common technique to assess such risks, yet existing approaches underperform in RAG due to black-box constraints and the absence of strong membership signals. In this paper, we identify a previously unexplored side channel in RAG systems: the generation budget, which controls the maximum number of tokens allowed in a generated response. Varying this budget reveals observable behavioral patterns between member and non-member queries, as members gain quality more rapidly with larger budgets. Building on this insight, we propose BudgetLeak, a novel membership inference attack that probes responses under different budgets and analyzes metric evolution via sequence modeling or clustering. Extensive experiments across four datasets, three LLM generators, and two retrievers demonstrate that BudgetLeak consistently outperforms existing baselines, while maintaining high efficiency and practical viability. Our findings reveal a previously overlooked data risk in RAG systems and highlight the need for new defenses.

Related papers

• RAG-CT: Mitigating Privacy Risks on Retrieval-Augmented Generation Systems via Scanning Prompt Distribution [16.269687907316843]
  Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for improving the quality of generated contents of Large Language Models (LLMs)<n>Recent studies have highlighted a critical vulnerability: adversaries can exploit the retrieval process to extract personally identifiable information (PII) from the underlying corpus.<n>We propose a novel defense, RAG-CT, that identifies malicious queries by analyzing their entropy and margin distributions and using a score-based detection method.
  arXiv  Detail & Related papers  (2026-09-14T14:44:39Z)
• Inference Cost Attacks for Retrieval-Augmented Large Language Models [21.602741677909915]
  We introduce the Retrieval-Augmented Inference Cost Attack (RA-ICA)<n>RA-ICA targets the computational cost of RAG-enhanced LLM systems by injecting malicious documents into external knowledge corpus.<n>We show that RA-ICA increases token consumption by up to 13.12 times with an over 90% success rate, without degrading the integrity of the generated answer.
  arXiv  Detail & Related papers  (2026-05-31T15:11:59Z)
• AIPO: Learning to Reason from Active Interaction [54.10819421625103]
  AIPO enables the policy model to proactively consult three functional collaborative agents when encountering reasoning bottlenecks.<n>AIPO consistently improves reasoning performance, generalizes robustly across different policy models and RLVR algorithms, and effectively expands the reasoning capability boundary of the policy model.
  arXiv  Detail & Related papers  (2026-05-08T19:06:55Z)
• Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation [50.87199039334856]
  Retrieval-Augmented Generation (RAG) has become a cornerstone of knowledge-intensive applications.<n>Recent studies show that knowledge-extraction attacks can recover sensitive knowledge-base content through maliciously crafted queries.<n>We introduce the first systematic benchmark for knowledge-extraction attacks on RAG systems.
  arXiv  Detail & Related papers  (2026-02-10T01:27:46Z)
• Connect the Dots: Knowledge Graph-Guided Crawler Attack on Retrieval-Augmented Generation Systems [14.028345839891855]
  Retrieval-augmented generation (RAG) systems integrate document retrieval with large language models.<n>RAG introduces a new privacy risk: adversaries can issue carefully crafted queries to exfiltrate sensitive content gradually.<n>We introduce RAGCRAWLER, which builds a knowledge graph to represent revealed information, and plans queries in semantic space that target unretrieved regions.
  arXiv  Detail & Related papers  (2026-01-22T05:59:42Z)
• SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems [53.51921540246166]
  Retrieval-Augmented Generation (RAG) techniques have become widely popular.<n>RAG involves the coupling of Large Language Models (LLMs) with domain-specific knowledge bases.<n>The proliferation of RAG has sparked concerns about data privacy.
  arXiv  Detail & Related papers  (2026-01-07T14:50:41Z)
• MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning Attacks [47.46936341268548]
  Retrieval-Augmented Generation (RAG) systems introduce a critical attack surface: corpus poisoning.<n>We propose MIRAGE, a novel multi-stage poisoning pipeline designed for strict black-box and query-agnostic environments.<n>Extensive experiments demonstrate that MIRAGE significantly outperforms existing baselines in both attack efficacy and stealthiness.
  arXiv  Detail & Related papers  (2025-12-09T06:38:16Z)
• SMA: Who Said That? Auditing Membership Leakage in Semi-Black-box RAG Controlling [50.66950115630554]
  Retrieval-Augmented Generation (RAG) and its Multimodal Retrieval-Augmented Generation (MRAG) significantly improve the knowledge coverage and contextual understanding of Large Language Models (LLMs)<n>However, retrieval and multimodal fusion obscure content provenance, rendering existing membership inference methods unable to reliably attribute generated outputs to pre-training, external retrieval, or user input, thus undermining privacy leakage accountability.<n>We propose the first Source-aware Membership Audit (SMA) that enables fine-grained source attribution of generated content in a semi-black-box setting with retrieval control capabilities.
  arXiv  Detail & Related papers  (2025-08-12T17:32:24Z)
• MrM: Black-Box Membership Inference Attacks against Multimodal RAG Systems [31.53306157650065]
  Multimodal retrieval-augmented generation (RAG) systems enhance large vision-language models by integrating cross-modal knowledge.<n>These knowledge databases may contain sensitive information that requires privacy protection.<n>MrM is the first black-box MIA framework targeted at multimodal RAG systems.
  arXiv  Detail & Related papers  (2025-06-09T03:48:50Z)
• Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey [92.36487127683053]
  Retrieval-Augmented Generation (RAG) is an advanced technique designed to address the challenges of Artificial Intelligence-Generated Content (AIGC)<n>RAG provides reliable and up-to-date external knowledge, reduces hallucinations, and ensures relevant context across a wide range of tasks.<n>Despite RAG's success and potential, recent studies have shown that the RAG paradigm also introduces new risks, including privacy concerns, adversarial attacks, and accountability issues.
  arXiv  Detail & Related papers  (2025-02-08T06:50:47Z)
• Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases [11.101624331624933]
  This paper presents a black-box attack to force a RAG system to leak its private knowledge base.<n>A relevance-based mechanism and an attacker-side open-source LLM favor the generation of effective queries to leak most of the (hidden) knowledge base.
  arXiv  Detail & Related papers  (2024-12-24T09:03:57Z)
• Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework [77.45983464131977]
  We focus on how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications.<n>Our research identifies two critical latent factors affecting RAG's confidence in its predictions.<n>We develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers.
  arXiv  Detail & Related papers  (2024-09-24T14:52:14Z)
• Trustworthiness in Retrieval-Augmented Generation Systems: A Survey [59.26328612791924]
  Retrieval-Augmented Generation (RAG) has quickly grown into a pivotal paradigm in the development of Large Language Models (LLMs) We propose a unified framework that assesses the trustworthiness of RAG systems across six key dimensions: factuality, robustness, fairness, transparency, accountability, and privacy.
  arXiv  Detail & Related papers  (2024-09-16T09:06:44Z)
• RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework [66.93260816493553]
  This paper introduces RAGEval, a framework designed to assess RAG systems across diverse scenarios.<n>With a focus on factual accuracy, we propose three novel metrics: Completeness, Hallucination, and Irrelevance.<n> Experimental results show that RAGEval outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples.
  arXiv  Detail & Related papers  (2024-08-02T13:35:11Z)
• Generating Is Believing: Membership Inference Attacks against Retrieval-Augmented Generation [9.73190366574692]
  Retrieval-Augmented Generation (RAG) is a technique that mitigates issues such as hallucinations and knowledge staleness in Large Language Models (LLMs) Existing research has demonstrated potential privacy risks associated with the LLMs of RAG. We present S$2$MIA, a underlineMembership underlineInference underlineAttack that utilizes the underlineSemantic underlineSimilarity between a given sample and the content generated by the RAG system.
  arXiv  Detail & Related papers  (2024-06-27T14:58:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.