Does Vec2Text Pose a New Corpus Poisoning Threat?

• URL: http://arxiv.org/abs/2410.06628v1
• Date: Wed, 09 Oct 2024 07:23:02 GMT
• Title: Does Vec2Text Pose a New Corpus Poisoning Threat?
• Authors: Shengyao Zhuang, Bevan Koopman, Guido Zuccon,
• Abstract summary: Vec2Text -- a method for text embedding inversion -- has raised serious privacy concerns for dense retrieval systems. In this paper, we take a new look at Vec2Text and investigate how much of a threat it poses to the different attacks of corpus poisoning. We show that under certain conditions, corpus poisoning with Vec2Text can pose a serious threat to dense retriever system integrity and user experience.
• Score: 30.78016941725723
• License:
• Abstract: The emergence of Vec2Text -- a method for text embedding inversion -- has raised serious privacy concerns for dense retrieval systems which use text embeddings. This threat comes from the ability for an attacker with access to embeddings to reconstruct the original text. In this paper, we take a new look at Vec2Text and investigate how much of a threat it poses to the different attacks of corpus poisoning, whereby an attacker injects adversarial passages into a retrieval corpus with the intention of misleading dense retrievers. Theoretically, Vec2Text is far more dangerous than previous attack methods because it does not need access to the embedding model's weights and it can efficiently generate many adversarial passages. We show that under certain conditions, corpus poisoning with Vec2Text can pose a serious threat to dense retriever system integrity and user experience by injecting adversarial passaged into top ranked positions. Code and data are made available at https://github.com/ielab/vec2text-corpus-poisoning

Related papers

• Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Models [0.0]
  Retrieval Augmented Generation (RAG) addresses this issue by combining Large Language Models with up-to-date information retrieval. This paper investigates prompt injection attacks on RAG, focusing on malicious objectives beyond misinformation. We build upon existing corpus poisoning techniques and propose a novel backdoor attack aimed at the fine-tuning process of the dense retriever component.
  arXiv  Detail & Related papers  (2024-10-18T14:02:34Z)
• Understanding and Mitigating the Threat of Vec2Text to Dense Retrieval Systems [28.175920880194223]
  We investigate factors related to embedding models that may impact text recoverability via Vec2Text. We propose a simple embedding transformation fix that guarantees equal ranking effectiveness while mitigating the recoverability risk.
  arXiv  Detail & Related papers  (2024-02-20T07:49:30Z)
• OrderBkd: Textual backdoor attack through repositioning [0.0]
  Third-party datasets and pre-trained machine learning models pose a threat to NLP systems. Existing backdoor attacks involve poisoning the data samples such as insertion of tokens or sentence paraphrasing. Our main difference from the previous work is that we use the reposition of a two words in a sentence as a trigger.
  arXiv  Detail & Related papers  (2024-02-12T14:53:37Z)
• Poisoning Retrieval Corpora by Injecting Adversarial Passages [79.14287273842878]
  We propose a novel attack for dense retrieval systems in which a malicious user generates a small number of adversarial passages. When these adversarial passages are inserted into a large retrieval corpus, we show that this attack is highly effective in fooling these systems. We also benchmark and compare a range of state-of-the-art dense retrievers, both unsupervised and supervised.
  arXiv  Detail & Related papers  (2023-10-29T21:13:31Z)
• Verifying the Robustness of Automatic Credibility Assessment [50.55687778699995]
  We show that meaning-preserving changes in input text can mislead the models. We also introduce BODEGA: a benchmark for testing both victim models and attack methods on misinformation detection tasks. Our experimental results show that modern large language models are often more vulnerable to attacks than previous, smaller solutions.
  arXiv  Detail & Related papers  (2023-03-14T16:11:47Z)
• Precise Zero-Shot Dense Retrieval without Relevance Labels [60.457378374671656]
  Hypothetical Document Embeddings(HyDE) is a zero-shot dense retrieval system. We show that HyDE significantly outperforms the state-of-the-art unsupervised dense retriever Contriever.
  arXiv  Detail & Related papers  (2022-12-20T18:09:52Z)
• Robust and Verifiable Information Embedding Attacks to Deep Neural Networks via Error-Correcting Codes [81.85509264573948]
  In the era of deep learning, a user often leverages a third-party machine learning tool to train a deep neural network (DNN) classifier. In an information embedding attack, an attacker is the provider of a malicious third-party machine learning tool. In this work, we aim to design information embedding attacks that are verifiable and robust against popular post-processing methods.
  arXiv  Detail & Related papers  (2020-10-26T17:42:42Z)
• Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder [78.01180944665089]
  This paper demonstrates a fatal vulnerability in natural language inference (NLI) and text classification systems. We present a 'backdoor poisoning' attack on NLP models.
  arXiv  Detail & Related papers  (2020-10-06T13:03:49Z)
• Humpty Dumpty: Controlling Word Meanings via Corpus Poisoning [29.181547214915238]
  We show that an attacker can control the "meaning" of new and existing words by changing their locations in the embedding space. An attack on the embedding can affect diverse downstream tasks, demonstrating for the first time the power of data poisoning in transfer learning scenarios.
  arXiv  Detail & Related papers  (2020-01-14T17:48:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.