Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrieval

• URL: http://arxiv.org/abs/2504.17884v1
• Date: Thu, 24 Apr 2025 18:46:11 GMT
• Title: Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrieval
• Authors: Yongkang Li, Panagiotis Eustratiadis, Simon Lupart, Evangelos Kanoulas,
• Abstract summary: We present an adversarial corpus attack that is fast and effective.<n>We consider attacks under both a white-box and a black-box setting.<n>Our method can generate successful adversarial examples in under two minutes per target document.
• Score: 15.141046442409953
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper concerns corpus poisoning attacks in dense information retrieval, where an adversary attempts to compromise the ranking performance of a search algorithm by injecting a small number of maliciously generated documents into the corpus. Our work addresses two limitations in the current literature. First, attacks that perform adversarial gradient-based word substitution search do so in the discrete lexical space, while retrieval itself happens in the continuous embedding space. We thus propose an optimization method that operates in the embedding space directly. Specifically, we train a perturbation model with the objective of maintaining the geometric distance between the original and adversarial document embeddings, while also maximizing the token-level dissimilarity between the original and adversarial documents. Second, it is common for related work to have a strong assumption that the adversary has prior knowledge about the queries. In this paper, we focus on a more challenging variant of the problem where the adversary assumes no prior knowledge about the query distribution (hence, unsupervised). Our core contribution is an adversarial corpus attack that is fast and effective. We present comprehensive experimental results on both in- and out-of-domain datasets, focusing on two related tasks: a top-1 attack and a corpus poisoning attack. We consider attacks under both a white-box and a black-box setting. Notably, our method can generate successful adversarial examples in under two minutes per target document; four times faster compared to the fastest gradient-based word substitution methods in the literature with the same hardware. Furthermore, our adversarial generation method generates text that is more likely to occur under the distribution of natural text (low perplexity), and is therefore more difficult to detect.

Related papers

• Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges [52.96987928118327]
  We find that embedding models for retrieval, rerankers, and large language model (LLM) relevance judges are vulnerable to content injection attacks.<n>We identify two primary threats: (1) inserting unrelated or harmful content within passages that still appear deceptively "relevant", and (2) inserting entire queries or key query terms into passages to boost their perceived relevance.<n>Our study systematically examines the factors that influence an attack's success, such as the placement of injected content and the balance between relevant and non-relevant material.
  arXiv  Detail & Related papers  (2025-01-30T18:02:15Z)
• Reproducing HotFlip for Corpus Poisoning Attacks in Dense Retrieval [14.799512604321363]
  HotFlip is a topical gradient-based word substitution method for attacking language models.<n>In this paper, we significantly boost the efficiency of HotFlip, reducing the adversarial generation process from 4 hours per document to only 15 minutes.<n>We also contribute experiments and analysis on two additional tasks: (1) transfer-based black-box attacks, and (2) query-agnostic attacks.
  arXiv  Detail & Related papers  (2025-01-08T19:29:33Z)
• Contextual Document Embeddings [77.22328616983417]
  We propose two complementary methods for contextualized document embeddings. First, an alternative contrastive learning objective that explicitly incorporates the document neighbors into the intra-batch contextual loss. Second, a new contextual architecture that explicitly encodes neighbor document information into the encoded representation.
  arXiv  Detail & Related papers  (2024-10-03T14:33:34Z)
• Adversarial Decoding: Generating Readable Documents for Adversarial Objectives [9.200635465485067]
  A new, generic text generation technique produces readable documents for different adversarial objectives.<n>Prior methods either produce easily detectable gibberish, or cannot handle objectives that include embedding similarity.<n>We measure the effectiveness of adversarial decoding for different objectives, including RAG poisoning, jailbreaking, and evasion of defensive filters.
  arXiv  Detail & Related papers  (2024-10-03T03:06:42Z)
• HQA-Attack: Toward High Quality Black-Box Hard-Label Adversarial Attack on Text [40.58680960214544]
  Black-box hard-label adversarial attack on text is a practical and challenging task. We propose a framework to generate high quality adversarial examples under the black-box hard-label attack scenarios, named HQA-Attack.
  arXiv  Detail & Related papers  (2024-02-02T10:06:43Z)
• Verifying the Robustness of Automatic Credibility Assessment [50.55687778699995]
  We show that meaning-preserving changes in input text can mislead the models. We also introduce BODEGA: a benchmark for testing both victim models and attack methods on misinformation detection tasks. Our experimental results show that modern large language models are often more vulnerable to attacks than previous, smaller solutions.
  arXiv  Detail & Related papers  (2023-03-14T16:11:47Z)
• A Tale of HodgeRank and Spectral Method: Target Attack Against Rank Aggregation Is the Fixed Point of Adversarial Game [153.74942025516853]
  The intrinsic vulnerability of the rank aggregation methods is not well studied in the literature. In this paper, we focus on the purposeful adversary who desires to designate the aggregated results by modifying the pairwise data. The effectiveness of the suggested target attack strategies is demonstrated by a series of toy simulations and several real-world data experiments.
  arXiv  Detail & Related papers  (2022-09-13T05:59:02Z)
• Learning-based Hybrid Local Search for the Hard-label Textual Attack [53.92227690452377]
  We consider a rarely investigated but more rigorous setting, namely hard-label attack, in which the attacker could only access the prediction label. Based on this observation, we propose a novel hard-label attack, called Learning-based Hybrid Local Search (LHLS) algorithm. Our LHLS significantly outperforms existing hard-label attacks regarding the attack performance as well as adversary quality.
  arXiv  Detail & Related papers  (2022-01-20T14:16:07Z)
• Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework [17.17479625646699]
  We propose a unified framework to craft textual adversarial samples. In this paper, we instantiate our framework with an attack algorithm named Textual Projected Gradient Descent (T-PGD)
  arXiv  Detail & Related papers  (2021-10-28T17:31:51Z)
• Semantic-Preserving Adversarial Text Attacks [85.32186121859321]
  We propose a Bigram and Unigram based adaptive Semantic Preservation Optimization (BU-SPO) method to examine the vulnerability of deep models. Our method achieves the highest attack success rates and semantics rates by changing the smallest number of words compared with existing methods.
  arXiv  Detail & Related papers  (2021-08-23T09:05:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.