ToxScreen: Detecting Whether an LLM Has Been Poisoned

• URL: http://arxiv.org/abs/2607.26849v1
• Date: Wed, 29 Jul 2026 12:35:58 GMT
• Title: ToxScreen: Detecting Whether an LLM Has Been Poisoned
• Abstract summary: We release ToxScreen, a benchmark of roughly 800 backdoored models spanning attack objectives, trigger mechanisms, poisoning rates, model scales, and backdoor training mechanisms.<n>We find that gradient-based prompt optimization fails in recovery, whereas a token look-up that ranks candidates by attack-success rate recovers the trigger wherever the backdoor is effective.<n>No method reliably surfaces every backdoor, but a broadly jailbreakable model is itself anomalous, a useful signal even when the exact trigger is not recovered.
• Score: 1.2598422062985144
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned. To evaluate whether a defender can recover such a trigger under realistic settings, we release ToxScreen, a benchmark of roughly 800 backdoored models spanning attack objectives, trigger mechanisms, poisoning rates, model scales, and backdoor training mechanisms. We also assert that the backdoors are high-quality: they achieve high attack success rates, generalize to unseen harmful inputs, and preserve clean-task performance. Scoring recovery of the planted trigger, we find that gradient-based prompt optimization fails in recovery, whereas a token look-up that ranks candidates by attack-success rate recovers the trigger wherever the backdoor is effective. To understand this more, we study the relationship between attack behaviors and the weights of an LLM. We find a phenomenon whereby backdoors operate via different mechanistic strategies than jailbreaks, allowing defenders to filter jailbreaks. Finally, no method reliably surfaces every backdoor, but a broadly jailbreakable model is itself anomalous, a useful signal even when the exact trigger is not recovered. We release all models and evaluation code

Related papers

• Backdoors in RLVR: Jailbreak Backdoors in LLMs From Verifiable Reward [16.80688807297612]
  Reinforcement Learning with Verifiable Rewards (RLVR) is an emerging paradigm that significantly boosts a Large Language Model's (LLM's) reasoning abilities.<n>We identify, for the first time, a latent vulnerability to backdoor attacks within the RLVR framework.<n>This attack can implant a backdoor without modifying the reward verifier by injecting a small amount of poisoning data into the training set.
  arXiv  Detail & Related papers  (2026-04-10T09:32:34Z)
• The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers [2.2374050209578864]
  We present a practical scanner for identifying sleeper agent-style backdoors in causal language models.<n>Our approach relies on two key findings: first, sleeper agents tend to memorize poisoning data, making it possible to leak backdoor examples.<n>We show that our method recovers working triggers across multiple backdoor scenarios and a broad range of models.
  arXiv  Detail & Related papers  (2026-02-03T04:17:21Z)
• Backdoor Unlearning by Linear Task Decomposition [69.91984435094157]
  Foundation models are highly susceptible to adversarial perturbations and targeted backdoor attacks.<n>Existing backdoor removal approaches rely on costly fine-tuning to override the harmful behavior.<n>This raises the question of whether backdoors can be removed without compromising the general capabilities of the models.
  arXiv  Detail & Related papers  (2025-10-16T16:18:07Z)
• A4O: All Trigger for One sample [10.78460062665304]
  We show that proposed backdoor defenders often rely on the assumption that triggers would appear in a unified way.<n>In this paper, we show that this naive assumption can create a loophole, allowing more sophisticated backdoor attacks to bypass.<n>We design a novel backdoor attack mechanism that incorporates multiple types of backdoor triggers, focusing on stealthiness and effectiveness.
  arXiv  Detail & Related papers  (2025-01-13T10:38:58Z)
• Data Free Backdoor Attacks [83.10379074100453]
  DFBA is a retraining-free and data-free backdoor attack without changing the model architecture.<n>We verify that our injected backdoor is provably undetectable and unchosen by various state-of-the-art defenses.<n>Our evaluation on multiple datasets demonstrates that our injected backdoor: 1) incurs negligible classification loss, 2) achieves 100% attack success rates, and 3) bypasses six existing state-of-the-art defenses.
  arXiv  Detail & Related papers  (2024-12-09T05:30:25Z)
• Transferring Backdoors between Large Language Models by Knowledge Distillation [2.9138150728729064]
  Backdoor Attacks have been a serious vulnerability against Large Language Models (LLMs) Previous methods only reveal such risk in specific models, or present tasks transferability after attacking the pre-trained phase. We propose ATBA, an adaptive transferable backdoor attack, which can effectively distill the backdoor of teacher LLMs into small models.
  arXiv  Detail & Related papers  (2024-08-19T10:39:45Z)
• Mitigating Backdoor Attack by Injecting Proactive Defensive Backdoor [63.84477483795964]
  Data-poisoning backdoor attacks are serious security threats to machine learning models. In this paper, we focus on in-training backdoor defense, aiming to train a clean model even when the dataset may be potentially poisoned. We propose a novel defense approach called PDB (Proactive Defensive Backdoor)
  arXiv  Detail & Related papers  (2024-05-25T07:52:26Z)
• Backdoor Attack with Sparse and Invisible Trigger [57.41876708712008]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. backdoor attack is an emerging yet threatening training-phase threat. We propose a sparse and invisible backdoor attack (SIBA)
  arXiv  Detail & Related papers  (2023-05-11T10:05:57Z)
• INK: Inheritable Natural Backdoor Attack Against Model Distillation [8.937026844871074]
  We introduce INK, an inheritable natural backdoor attack that targets model distillation. INK employs image variance as a backdoor trigger and enables both clean-image and clean-label attacks. For instance, INK maintains an attack success rate of over 98% post-distillation, compared to an average success rate of 1.4% for existing methods.
  arXiv  Detail & Related papers  (2023-04-21T14:35:47Z)
• Stealthy Backdoor Attack for Code Models [19.272856932095966]
  Existing backdoor attacks on code models use unstealthy and easy-to-detect triggers. This paper aims to investigate the vulnerability of code models with stealthy backdoor attacks. We find that around 85% of adaptive triggers in AFRAIDOOR bypass the detection in the defense process.
  arXiv  Detail & Related papers  (2023-01-06T13:15:42Z)
• Black-box Detection of Backdoor Attacks with Limited Information and Data [56.0735480850555]
  We propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models.
  arXiv  Detail & Related papers  (2021-03-24T12:06:40Z)
• Clean-Label Backdoor Attacks on Video Recognition Models [87.46539956587908]
  We show that image backdoor attacks are far less effective on videos. We propose the use of a universal adversarial trigger as the backdoor trigger to attack video recognition models. Our proposed backdoor attack is resistant to state-of-the-art backdoor defense/detection methods.
  arXiv  Detail & Related papers  (2020-03-06T04:51:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.