Cordyceps: Covert Control Attacks on LLMs via Data Poisoning

• URL: http://arxiv.org/abs/2605.26595v1
• Date: Tue, 26 May 2026 06:28:30 GMT
• Title: Cordyceps: Covert Control Attacks on LLMs via Data Poisoning
• Abstract summary: Large language models (LLMs) are often fine-tuned on uncurated text datasets that can poison adversaries.<n>We propose a data poisoning method that teaches an LLM an information hiding scheme reliably and stealthily.<n>The induced hiding scheme can encode and decode arbitrary malicious instructions, thus revealing a new and subtle poisoning-induced vulnerability: covert control attacks.
• Score: 7.104161390387404
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison. Existing poisoning attacks primarily rely on fixed trigger phrases that defenses such as outlier detection, clean-data regularization, or online monitoring can neutralize. In this paper, we propose a data poisoning method that teaches an LLM an information hiding scheme reliably and stealthily through semantic associations between shared knowledge such as facts or concepts and attacker-chosen phrases. The induced hiding scheme can encode and decode arbitrary malicious instructions, thus revealing a new and subtle poisoning-induced vulnerability: covert control attacks. We precisely characterize covert control attacks and evaluate them across $5$ LLMs, $3$ backdoor defenses, and $4$ prompt injection defenses. With a small poisoned fraction, covert control attacks outperform heuristic-based prompt injection attacks in average attack success rate by about $40\%$ relative to clean fine-tuned models. They also circumvent defenses based on detection and fine-tuning, maintaining up to $93\%$ attack success rate after backdoor defenses and up to $98\%$ after prompt injection defenses.

Related papers

• P2P: A Poison-to-Poison Remedy for Reliable Backdoor Defense in LLMs [49.908234151374785]
  During fine-tuning, large language models (LLMs) are increasingly vulnerable to data-poisoning backdoor attacks.<n>We propose Poison-to-Poison (P2P), a general and effective backdoor defense algorithm.<n>We show that P2P can neutralize malicious backdoors while preserving task performance.
  arXiv  Detail & Related papers  (2025-10-06T05:45:23Z)
• Backdoor-Powered Prompt Injection Attacks Nullify Defense Methods [95.54363609024847]
  Large language models (LLMs) are vulnerable to prompt injection attacks.<n>In this paper, we explore more vicious attacks that nullify the prompt injection defense methods.<n> backdoor-powered prompt injection attacks are more harmful than previous prompt injection attacks.
  arXiv  Detail & Related papers  (2025-10-04T07:11:11Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• A Study of Backdoors in Instruction Fine-tuned Language Models [16.10608633005216]
  Backdoor data poisoning is a serious security concern due to the evasive nature of such attacks. Such backdoor attacks can: alter response sentiment, violate censorship, over-refuse (invoke censorship for legitimate queries), inject false content, or trigger nonsense responses (hallucinations)
  arXiv  Detail & Related papers  (2024-06-12T00:01:32Z)
• SEEP: Training Dynamics Grounds Latent Representation Search for Mitigating Backdoor Poisoning Attacks [53.28390057407576]
  Modern NLP models are often trained on public datasets drawn from diverse sources. Data poisoning attacks can manipulate the model's behavior in ways engineered by the attacker. Several strategies have been proposed to mitigate the risks associated with backdoor attacks.
  arXiv  Detail & Related papers  (2024-05-19T14:50:09Z)
• Certified Robustness to Clean-Label Poisoning Using Diffusion Denoising [56.04951180983087]
  We present a certified defense to clean-label poisoning attacks under $ell$-norm.<n>Inspired by the adversarial robustness achieved by $randomized$ $smoothing, we show how an off-the-shelf diffusion denoising model can sanitize the tampered training data.
  arXiv  Detail & Related papers  (2024-03-18T17:17:07Z)
• Learning to Poison Large Language Models for Downstream Manipulation [12.521338629194503]
  This work identifies additional security risks in Large Language Models (LLMs) by designing a new data poisoning attack tailored to exploit the supervised fine-tuning process.<n>We propose a novel gradient-guided backdoor trigger learning (GBTL) algorithm to identify adversarial triggers efficiently.<n>We propose two defense strategies against data poisoning attacks, including in-context learning (ICL) and continuous learning (CL)
  arXiv  Detail & Related papers  (2024-02-21T01:30:03Z)
• Beating Backdoor Attack at Its Own Game [10.106197319676294]
  Deep neural networks (DNNs) are vulnerable to backdoor attack.<n>Existing defense methods have greatly reduced attack success rate.<n>We propose a highly effective framework which injects non-adversarial backdoors targeting poisoned samples.
  arXiv  Detail & Related papers  (2023-07-28T13:07:42Z)
• Narcissus: A Practical Clean-Label Backdoor Attack with Limited Information [22.98039177091884]
  "Clean-label" backdoor attacks require knowledge of the entire training set to be effective. This paper provides an algorithm to mount clean-label backdoor attacks based only on the knowledge of representative examples from the target class. Our attack works well across datasets and models, even when the trigger presents in the physical world.
  arXiv  Detail & Related papers  (2022-04-11T16:58:04Z)
• What Doesn't Kill You Makes You Robust(er): Adversarial Training against Poisons and Backdoors [57.040948169155925]
  We extend the adversarial training framework to defend against (training-time) poisoning and backdoor attacks. Our method desensitizes networks to the effects of poisoning by creating poisons during training and injecting them into training batches. We show that this defense withstands adaptive attacks, generalizes to diverse threat models, and incurs a better performance trade-off than previous defenses.
  arXiv  Detail & Related papers  (2021-02-26T17:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.