Mind the Hook: Source-Level Auditing of Privacy Defenses in Retrieval-Augmented Generation

• URL: http://arxiv.org/abs/2608.09001v1
• Date: Mon, 10 Aug 2026 01:40:25 GMT
• Title: Mind the Hook: Source-Level Auditing of Privacy Defenses in Retrieval-Augmented Generation
• Abstract summary: Black-box privacy scores for retrieval-augmented generation (RAG) are difficult to interpret unless the defense's active pipeline hook is known.<n>We propose an active-path audit: inventory source-level hooks over retrieval, retrieved content, and generation; map each metric to the leakage channel it observes; and validate generated-text effects with exact-match canaries.
• Score: 0.30586855806896046
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Black-box privacy scores for retrieval-augmented generation (RAG) are difficult to interpret unless the audited defense's active pipeline hook is known. We propose an active-path audit: inventory source-level hooks over retrieval, retrieved content, and generation; map each metric to the leakage channel it observes; and validate generated-text effects with exact-match canaries. In our benchmark reimplementations, the DP-style defenses modify retrieval scores only: their generation hooks are TODO-flagged stubs that return responses unchanged. This active path explains why they affect membership-inference behavior but track No-Defense on generated-text named-entity leakage, measured by NEL_strict. By contrast, the end-to-end LPRAG path is canary-validated on the email channel, recovering 53/150 canaries under No-Defense and 0/150 under LPRAG. These findings concern our reimplementations on our stack, not released defenses or defense families; the contribution is a methodology and case study, not a universal ranking

Related papers

• Labels Are Not Endpoints: Treatment Leakage and Construct Validity in MCP Agent Security Evaluation [0.0]
  We audit a preserved campaign by tracing 10,200 execution rows to 180 model-bound requests, 45 semantic requests, and 15 observable stimuli.<n>We contribute a seven-link Integrity Chain and an executable, scope-bounded endpoint-integrity linter.
  arXiv  Detail & Related papers  (2026-08-13T06:44:40Z)
• Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit [8.665845754660458]
  AI-generated video detection GenVidBench and AIGVDBench are the de facto leaderboards.<n>Most evaluation protocols leave uncontrolled confounds that can inflate reported generalizations.<n>A 20-paper survey finds none applying all six standard controls that would catch this.<n>We combine them into an audited protocol and apply it to six representative feature sources.
  arXiv  Detail & Related papers  (2026-06-30T00:29:53Z)
• Auditing Privacy in Multi-Tenant RAG under Account Collusion [1.253312107729806]
  Multi-tenant retrieval-augmented generation services advertise per-account differential privacy as the operative leakage boundary.<n>We identify same-index multi-account collusion as a privacy-boundary failure.<n>We design the first audit protocol that operates against unmodified RAG deployments.
  arXiv  Detail & Related papers  (2026-05-19T13:41:59Z)
• Needle-in-RAG: Prompt-Conditioned Character-Level Traceback of Poisoned Spans in Retrieved Evidence [4.039934762896615]
  RAGCharacter is a two-pass forensic framework that localizes the responsible retrieved span for a concrete misgeneration event.<n>We study black-box character-level poison traceback in RAG and present RAGCharacter, a two-pass forensic framework that localizes the responsible retrieved span for a concrete misgeneration event.
  arXiv  Detail & Related papers  (2026-05-03T08:42:29Z)
• Detecting RAG Extraction Attack via Dual-Path Runtime Integrity Game [23.794446184684755]
  Retrieval-Augmented Generation (RAG) systems augment large language models with external knowledge, yet introduce a critical security vulnerability RAG Knowledge Base Leakage.<n>We propose CanaryRAG, a runtime defense mechanism inspired by stack canaries in software security.<n>CanaryRAG embeds carefully designed canary tokens into retrieved chunks and reformulates RAG extraction defense as a dual-path runtime integrity game.
  arXiv  Detail & Related papers  (2026-04-12T16:28:09Z)
• The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search [58.8834056209347]
  Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs.<n>We introduce the Correlated Knowledge Attack Agent (CKA-Agent), a dynamic framework that reframes jailbreaking as an adaptive, tree-structured exploration of the target model's knowledge base.
  arXiv  Detail & Related papers  (2025-12-01T07:05:23Z)
• SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking [58.475471437150674]
  We propose sequential watermarking for soft prompts (SWAP)<n>SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes.<n>Experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-11-05T13:48:48Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning [58.57194301645823]
  Large language models (LLMs) are increasingly integrated into real-world personalized applications.<n>The valuable and often proprietary nature of the knowledge bases used in RAG introduces the risk of unauthorized usage by adversaries.<n>Existing methods that can be generalized as watermarking techniques to protect these knowledge bases typically involve poisoning or backdoor attacks.<n>We propose name for harmless' copyright protection of knowledge bases.
  arXiv  Detail & Related papers  (2025-02-10T09:15:56Z)
• Certifiably Robust RAG against Retrieval Corruption [58.677292678310934]
  Retrieval-augmented generation (RAG) has been shown vulnerable to retrieval corruption attacks. In this paper, we propose RobustRAG as the first defense framework against retrieval corruption attacks.
  arXiv  Detail & Related papers  (2024-05-24T13:44:25Z)
• Lazy Layers to Make Fine-Tuned Diffusion Models More Traceable [70.77600345240867]
  A novel arbitrary-in-arbitrary-out (AIAO) strategy makes watermarks resilient to fine-tuning-based removal. Unlike the existing methods of designing a backdoor for the input/output space of diffusion models, in our method, we propose to embed the backdoor into the feature space of sampled subpaths. Our empirical studies on the MS-COCO, AFHQ, LSUN, CUB-200, and DreamBooth datasets confirm the robustness of AIAO.
  arXiv  Detail & Related papers  (2024-05-01T12:03:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.