Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Reference

• URL: http://arxiv.org/abs/2603.13461v1
• Date: Fri, 13 Mar 2026 17:09:37 GMT
• Title: Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Reference
• Abstract summary: Backdoor attacks pose severe security threats to large language models (LLMs)<n>We propose a new framework for instruction-tuned LLM without prior trigger knowledge or clean references.<n>We produce purified models that withstand diverse backdoor attacks and threat models while preserving generative capability.
• Score: 15.215130286922564
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Backdoor attacks pose severe security threats to large language models (LLMs), where a model behaves normally under benign inputs but produces malicious outputs when a hidden trigger appears. Existing backdoor removal methods typically assume prior knowledge of triggers, access to a clean reference model, or rely on aggressive finetuning configurations, and are often limited to classification tasks. However, such assumptions fall apart in real-world instruction-tuned LLM settings. In this work, we propose a new framework for purifying instruction-tuned LLM without any prior trigger knowledge or clean references. Through systematic sanity checks, we find that backdoor associations are redundantly encoded across MLP layers, while attention modules primarily amplify trigger signals without establishing the behavior. Leveraging this insight, we shift the focus from isolating specific backdoor triggers to cutting off the trigger-behavior associations, and design an immunization-inspired elimination approach: by constructing multiple synthetic backdoored variants of the given suspicious model, each trained with different malicious trigger-behavior pairs, and contrasting them with their clean counterparts. The recurring modifications across variants reveal a shared "backdoor signature"-analogous to antigens in a virus. Guided by this signature, we neutralize highly suspicious components in LLM and apply lightweight finetuning to restore its fluency, producing purified models that withstand diverse backdoor attacks and threat models while preserving generative capability.

Related papers

• Not All Tokens Are Equal: Region-Aware Consistency Repair of Backdoors in MLLMs [19.117078494342714]
  Existing model-level backdoor removal methods show limited effectiveness on MLLMs.<n>We present RACER, a model-level repair framework motivated by a key observation: backdoors induce abnormal layer-to-layer evolution in internal representations.<n> RACER decomposes the fused representation into visual and textual token regions, normalizes their layer-wise inconsistency separately, and recomposes them using modality-aware weights over a deep-layer window.
  arXiv  Detail & Related papers  (2026-08-25T10:10:01Z)
• Curvature-Guided Module Localization for Low-Rank Detoxification of Backdoored Large Language Models [9.402151821242951]
  Backdoor attacks pose a serious threat to large language models (LLMs)<n>We propose a mechanistically guided weight-space repair framework that first localizes modules involved in propagating trigger-induced behavior.<n>Results show that the proposed approach substantially suppresses trigger-conditioned malicious responses while preserving benign model behavior.
  arXiv  Detail & Related papers  (2026-06-29T20:40:21Z)
• ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety [54.4092272526747]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success in cross-modal understanding and generation, yet their deployment is threatened by critical safety vulnerabilities.<n>We propose ProjLens, an interpretability framework designed to demystify MLLMs backdoors.
  arXiv  Detail & Related papers  (2026-04-21T04:52:38Z)
• Self-Purification Mitigates Backdoors in Multimodal Diffusion Language Models [74.1970982768771]
  We show that well-established data-poisoning pipelines can successfully implant backdoors into MDLMs.<n>We introduce a backdoor defense framework for MDLMs named DiSP (Diffusion Self-Purification)
  arXiv  Detail & Related papers  (2026-02-24T15:47:52Z)
• Backdoor Unlearning by Linear Task Decomposition [69.91984435094157]
  Foundation models are highly susceptible to adversarial perturbations and targeted backdoor attacks.<n>Existing backdoor removal approaches rely on costly fine-tuning to override the harmful behavior.<n>This raises the question of whether backdoors can be removed without compromising the general capabilities of the models.
  arXiv  Detail & Related papers  (2025-10-16T16:18:07Z)
• From Poisoned to Aware: Fostering Backdoor Self-Awareness in LLMs [27.723404842086072]
  Large Language Models (LLMs) can acquire deceptive behaviors through backdoor attacks.<n>Existing safety training methods fail to address this vulnerability.<n>We propose a novel post-training framework that cultivates self-awareness of backdoor risks.
  arXiv  Detail & Related papers  (2025-10-05T03:55:24Z)
• Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution [49.78359632298156]
  Large language models (LLMs) have seen significant advancements, achieving superior performance in various Natural Language Processing (NLP) tasks.<n> backdoor attacks, where models behave normally for standard queries but generate harmful responses or unintended output when specific triggers are activated.<n>We present LETHE, a novel method to eliminate backdoor behaviors from LLMs through knowledge dilution.
  arXiv  Detail & Related papers  (2025-08-28T17:05:18Z)
• Robust Anti-Backdoor Instruction Tuning in LVLMs [53.766434746801366]
  We introduce a lightweight, certified-agnostic defense framework for large visual language models (LVLMs)<n>Our framework finetunes only adapter modules and text embedding layers under instruction tuning.<n>Experiments against seven attacks on Flickr30k and MSCOCO demonstrate that ours reduces their attack success rate to nearly zero.
  arXiv  Detail & Related papers  (2025-06-04T01:23:35Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Neutralizing Backdoors through Information Conflicts for Large Language Models [20.6331157117675]
  We present a novel method to eliminate backdoor behaviors from large language models (LLMs)<n>We leverage a lightweight dataset to train a conflict model, which is then merged with the backdoored model to neutralize malicious behaviors.<n>We can reduce the attack success rate of advanced backdoor attacks by up to 98% while maintaining over 90% clean data accuracy.
  arXiv  Detail & Related papers  (2024-11-27T12:15:22Z)
• CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization [7.282200564983221]
  Large Language Models (LLMs) are vulnerable to backdoor attacks that manipulate outputs via hidden triggers.<n>We propose Internal Consistency Regularization (CROW), a defense leveraging the observation that backdoored models exhibit unstable layer-wise hidden representations when triggered.<n>CROW enforces consistency across layers via adversarial perturbations and regularization during finetuning, neutralizing backdoors without requiring clean reference models or trigger knowledge--only a small clean dataset.
  arXiv  Detail & Related papers  (2024-11-18T07:52:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.