Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases

• URL: http://arxiv.org/abs/2412.18295v2
• Date: Sun, 29 Dec 2024 21:25:04 GMT
• Title: Pirates of the RAG: Adaptively Attacking LLMs to Leak Knowledge Bases
• Authors: Christian Di Maio, Cristian Cosci, Marco Maggini, Valentina Poggioni, Stefano Melacci,
• Abstract summary: This paper presents a black-box attack to force a RAG system to leak its private knowledge base.<n>A relevance-based mechanism and an attacker-side open-source LLM favor the generation of effective queries to leak most of the (hidden) knowledge base.
• Score: 11.101624331624933
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The growing ubiquity of Retrieval-Augmented Generation (RAG) systems in several real-world services triggers severe concerns about their security. A RAG system improves the generative capabilities of a Large Language Models (LLM) by a retrieval mechanism which operates on a private knowledge base, whose unintended exposure could lead to severe consequences, including breaches of private and sensitive information. This paper presents a black-box attack to force a RAG system to leak its private knowledge base which, differently from existing approaches, is adaptive and automatic. A relevance-based mechanism and an attacker-side open-source LLM favor the generation of effective queries to leak most of the (hidden) knowledge base. Extensive experimentation proves the quality of the proposed algorithm in different RAG pipelines and domains, comparing to very recent related approaches, which turn out to be either not fully black-box, not adaptive, or not based on open-source models. The findings from our study remark the urgent need for more robust privacy safeguards in the design and deployment of RAG systems.

Related papers

• Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generation [60.81109086640437]
  We propose a novel framework called Federated Retrieval-Augmented Generation (FedE4RAG) FedE4RAG facilitates collaborative training of client-side RAG retrieval models. We apply homomorphic encryption within federated learning to safeguard model parameters.
  arXiv  Detail & Related papers  (2025-04-27T04:26:02Z)
• ControlNET: A Firewall for RAG-based LLM System [9.362574883495927]
  Retrieval-Augmented Generation (RAG) has significantly enhanced the factual accuracy and domain adaptability of Large Language Models (LLMs) RAG mitigates hallucinations by integrating external knowledge, yet introduces privacy risk and security risk, notably data breaching risk and data poisoning risk. In this paper, we propose an AI firewall, ControlNET, designed to safeguard RAG-based LLM systems from these vulnerabilities.
  arXiv  Detail & Related papers  (2025-04-13T14:18:35Z)
• The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systems [8.347617177093056]
  We introduce a realistic black-box attack scenario based on the RAG paradox, where RAG systems inadvertently expose vulnerabilities while attempting to enhance trustworthiness. Because RAG systems reference external documents during response generation, our attack targets these sources without requiring internal access. Our approach first identifies the external sources disclosed by RAG systems and then automatically generates poisoned documents with misinformation designed to match these sources.
  arXiv  Detail & Related papers  (2025-02-28T12:32:53Z)
• MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks [109.53357276796655]
  Multimodal large language models (MLLMs) equipped with Retrieval Augmented Generation (RAG) RAG enhances MLLMs by grounding responses in query-relevant external knowledge. This reliance poses a critical yet underexplored safety risk: knowledge poisoning attacks. We propose MM-PoisonRAG, a novel knowledge poisoning attack framework with two attack strategies.
  arXiv  Detail & Related papers  (2025-02-25T04:23:59Z)
• Towards Trustworthy Retrieval Augmented Generation for Large Language Models: A Survey [92.36487127683053]
  Retrieval-Augmented Generation (RAG) is an advanced technique designed to address the challenges of Artificial Intelligence-Generated Content (AIGC) RAG provides reliable and up-to-date external knowledge, reduces hallucinations, and ensures relevant context across a wide range of tasks. Despite RAG's success and potential, recent studies have shown that the RAG paradigm also introduces new risks, including privacy concerns, adversarial attacks, and accountability issues.
  arXiv  Detail & Related papers  (2025-02-08T06:50:47Z)
• RAG-Thief: Scalable Extraction of Private Data from Retrieval-Augmented Generation Applications with Agent-based Attacks [18.576435409729655]
  We propose an agent-based automated privacy attack called RAG-Thief. It can extract a scalable amount of private data from the private database used in RAG applications. Our findings highlight the privacy vulnerabilities in current RAG applications and underscore the pressing need for stronger safeguards.
  arXiv  Detail & Related papers  (2024-11-21T13:18:03Z)
• HijackRAG: Hijacking Attacks against Retrieval-Augmented Large Language Models [18.301965456681764]
  We reveal a novel vulnerability, the retrieval prompt hijack attack (HijackRAG) HijackRAG enables attackers to manipulate the retrieval mechanisms of RAG systems by injecting malicious texts into the knowledge database. We propose both black-box and white-box attack strategies tailored to different levels of the attacker's knowledge.
  arXiv  Detail & Related papers  (2024-10-30T09:15:51Z)
• Trustworthiness in Retrieval-Augmented Generation Systems: A Survey [59.26328612791924]
  Retrieval-Augmented Generation (RAG) has quickly grown into a pivotal paradigm in the development of Large Language Models (LLMs) We propose a unified framework that assesses the trustworthiness of RAG systems across six key dimensions: factuality, robustness, fairness, transparency, accountability, and privacy.
  arXiv  Detail & Related papers  (2024-09-16T09:06:44Z)
• ConfusedPilot: Confused Deputy Risks in RAG-based LLMs [2.423202571519879]
  We introduce ConfusedPilot, a class of security vulnerabilities of RAG systems that confuse Copilot and cause integrity and confidentiality violations in its responses. This study highlights the security vulnerabilities in today's RAG-based systems and proposes design guidelines to secure future RAG-based systems.
  arXiv  Detail & Related papers  (2024-08-09T05:20:05Z)
• Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation [64.7982176398485]
  Retrieval-augmented generation (RAG) has demonstrated effectiveness in mitigating the hallucination problem of large language models (LLMs) We propose DPA-RAG, a universal framework designed to align diverse knowledge preferences within RAG systems.
  arXiv  Detail & Related papers  (2024-06-26T18:26:53Z)
• "Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models [74.05368440735468]
  Retrieval-Augmented Generative (RAG) models enhance Large Language Models (LLMs) In this paper, we demonstrate a security threat where adversaries can exploit the openness of these knowledge bases.
  arXiv  Detail & Related papers  (2024-06-26T05:36:23Z)
• The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG) [56.67603627046346]
  Retrieval-augmented generation (RAG) is a powerful technique to facilitate language model with proprietary and private data. In this work, we conduct empirical studies with novel attack methods, which demonstrate the vulnerability of RAG systems on leaking the private retrieval database.
  arXiv  Detail & Related papers  (2024-02-23T18:35:15Z)
• ActiveRAG: Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents [49.30553350788524]
  Retrieval-Augmented Generation (RAG) enables Large Language Models (LLMs) to leverage external knowledge. Existing RAG models often treat LLMs as passive recipients of information. We introduce ActiveRAG, a multi-agent framework that mimics human learning behavior.
  arXiv  Detail & Related papers  (2024-02-21T06:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.