RAG with Differential Privacy

• URL: http://arxiv.org/abs/2412.19291v2
• Date: Wed, 22 Jan 2025 14:50:33 GMT
• Title: RAG with Differential Privacy
• Authors: Nicolas Grislain,
• Abstract summary: Retrieval-Augmented Generation (RAG) has emerged as the dominant technique to provide large language models with fresh and relevant context.<n>The integration of external documents into the generation process raises significant privacy concerns.<n>This paper explores a practical solution to this problem suitable to general knowledge extraction from personal data.
• Score: 3.009591302286514
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-Augmented Generation (RAG) has emerged as the dominant technique to provide \emph{Large Language Models} (LLM) with fresh and relevant context, mitigating the risk of hallucinations and improving the overall quality of responses in environments with large and fast moving knowledge bases. However, the integration of external documents into the generation process raises significant privacy concerns. Indeed, when added to a prompt, it is not possible to guarantee a response will not inadvertently expose confidential data, leading to potential breaches of privacy and ethical dilemmas. This paper explores a practical solution to this problem suitable to general knowledge extraction from personal data. It shows \emph{differentially private token generation} is a viable approach to private RAG.

Related papers

• Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generation [26.573578326262307]
  Privacy-Aware Decoding (PAD) is a lightweight, inference-time defense that adaptively injects calibrated Gaussian noise into token logits during generation.<n>PAD integrates confidence-based screening to selectively protect high-risk tokens, efficient sensitivity estimation to minimize unnecessary noise, and context-aware noise calibration to balance privacy with generation quality.<n>Our work takes an important step toward mitigating privacy risks in RAG via decoding strategies, paving the way for universal and scalable privacy solutions in sensitive domains.
  arXiv  Detail & Related papers  (2025-08-05T05:22:13Z)
• Fine-Grained Privacy Extraction from Retrieval-Augmented Generation Systems via Knowledge Asymmetry Exploitation [15.985529058573912]
  Retrieval-augmented generation (RAG) systems enhance large language models (LLMs) by integrating external knowledge bases.<n>Existing privacy attacks on RAG systems can trigger data leakage but often fail to accurately isolate knowledge-base-derived sentences within mixed responses.<n>This paper presents a novel black-box attack framework that exploits knowledge asymmetry between RAG and standard LLMs to achieve fine-grained privacy extraction.
  arXiv  Detail & Related papers  (2025-07-31T03:50:16Z)
• Beyond Text: Unveiling Privacy Vulnerabilities in Multi-modal Retrieval-Augmented Generation [17.859942323017133]
  We provide the first systematic analysis of MRAG privacy vulnerabilities across vision-language and speech-language modalities.<n>Our experiments reveal that LMMs can both directly generate outputs resembling retrieved content and produce descriptions that indirectly expose sensitive information.
  arXiv  Detail & Related papers  (2025-05-20T05:37:22Z)
• A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage [77.83757117924995]
  We propose a new framework that evaluates re-identification attacks to quantify individual privacy risks upon data release. Our approach shows that seemingly innocuous auxiliary information can be used to infer sensitive attributes like age or substance use history from sanitized data.
  arXiv  Detail & Related papers  (2025-04-28T01:16:27Z)
• Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generation [60.81109086640437]
  We propose a novel framework called Federated Retrieval-Augmented Generation (FedE4RAG) FedE4RAG facilitates collaborative training of client-side RAG retrieval models. We apply homomorphic encryption within federated learning to safeguard model parameters.
  arXiv  Detail & Related papers  (2025-04-27T04:26:02Z)
• Learning to Erase Private Knowledge from Multi-Documents for Retrieval-Augmented Large Language Models [30.143809176910185]
  This paper introduces the privacy erasure task for Retrieval-Augmented Generation (RAG) We first construct a global knowledge graph to identify potential knowledge across documents, aiming to defend against de-anonymization attacks. Experiments on four QA datasets demonstrate that Eraser4RAG superior erase performance than GPT-4o.
  arXiv  Detail & Related papers  (2025-04-14T06:10:31Z)
• Privacy-Preserving Retrieval Augmented Generation with Differential Privacy [25.896416088293908]
  Retrieval augmented generation (RAG) assists large language models (LLMs) by directly providing relevant information from external knowledge sources.<n>RAG outputs risk leaking sensitive information from the external data source.<n>In this work, we explore RAG under differential privacy (DP), a formal guarantee of data privacy.
  arXiv  Detail & Related papers  (2024-12-06T01:20:16Z)
• RAG-Thief: Scalable Extraction of Private Data from Retrieval-Augmented Generation Applications with Agent-based Attacks [18.576435409729655]
  We propose an agent-based automated privacy attack called RAG-Thief. It can extract a scalable amount of private data from the private database used in RAG applications. Our findings highlight the privacy vulnerabilities in current RAG applications and underscore the pressing need for stronger safeguards.
  arXiv  Detail & Related papers  (2024-11-21T13:18:03Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories. We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds. State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data [51.41288763521186]
  Retrieval-augmented generation (RAG) enhances the outputs of language models by integrating relevant information retrieved from external knowledge sources. RAG systems may face severe privacy risks when retrieving private data. We propose using synthetic data as a privacy-preserving alternative for the retrieval data.
  arXiv  Detail & Related papers  (2024-06-20T22:53:09Z)
• The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG) [56.67603627046346]
  Retrieval-augmented generation (RAG) is a powerful technique to facilitate language model with proprietary and private data. In this work, we conduct empirical studies with novel attack methods, which demonstrate the vulnerability of RAG systems on leaking the private retrieval database.
  arXiv  Detail & Related papers  (2024-02-23T18:35:15Z)
• InferDPT: Privacy-Preserving Inference for Black-box Large Language Model [66.07752875835506]
  InferDPT is the first practical framework for the privacy-preserving Inference of black-box LLMs. RANTEXT is a novel differential privacy mechanism integrated into the perturbation module of InferDPT.
  arXiv  Detail & Related papers  (2023-10-18T18:00:11Z)
• A Unified View of Differentially Private Deep Generative Modeling [60.72161965018005]
  Data with privacy concerns comes with stringent regulations that frequently prohibited data access and data sharing. Overcoming these obstacles is key for technological progress in many real-world application scenarios that involve privacy sensitive data. Differentially private (DP) data publishing provides a compelling solution, where only a sanitized form of the data is publicly released.
  arXiv  Detail & Related papers  (2023-09-27T14:38:16Z)
• How Do Input Attributes Impact the Privacy Loss in Differential Privacy? [55.492422758737575]
  We study the connection between the per-subject norm in DP neural networks and individual privacy loss. We introduce a novel metric termed the Privacy Loss-Input Susceptibility (PLIS) which allows one to apportion the subject's privacy loss to their input attributes.
  arXiv  Detail & Related papers  (2022-11-18T11:39:03Z)
• Privacy-Preserving Distributed Expectation Maximization for Gaussian Mixture Model using Subspace Perturbation [4.2698418800007865]
  federated learning is motivated by the privacy concern as it does not allow to transmit private data but only intermediate updates. We propose a fully decentralized privacy-preserving solution, which is able to securely compute the updates in each step. Numerical validation shows that the proposed approach has superior performance compared to the existing approach in terms of both the accuracy and privacy level.
  arXiv  Detail & Related papers  (2022-09-16T09:58:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.