Curvature-Guided Module Localization for Low-Rank Detoxification of Backdoored Large Language Models

• URL: http://arxiv.org/abs/2606.30899v1
• Date: Mon, 29 Jun 2026 20:40:21 GMT
• Title: Curvature-Guided Module Localization for Low-Rank Detoxification of Backdoored Large Language Models
• Abstract summary: Backdoor attacks pose a serious threat to large language models (LLMs)<n>We propose a mechanistically guided weight-space repair framework that first localizes modules involved in propagating trigger-induced behavior.<n>Results show that the proposed approach substantially suppresses trigger-conditioned malicious responses while preserving benign model behavior.
• Score: 9.402151821242951
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Backdoor attacks pose a serious threat to large language models (LLMs) by causing otherwise benign systems to produce attacker-specified malicious behavior when a hidden trigger is present. In this work, we study post hoc detoxification of backdoored LLMs in a practical setting where the defender has access to the poisoned model but does not wish to retrain the full network from scratch. We propose a mechanistically guided weight-space repair framework that first localizes modules involved in propagating trigger-induced behavior using activation patching and Fisher/K-FAC curvature analysis, and then applies targeted low-rank repair to only the most influential modules. We evaluate the method on poisoned variants of \texttt{Llama-3.2-1B-Instruct} with triggers inserted at the beginning, middle, and end of otherwise benign prompts. Results show that the proposed approach substantially suppresses trigger-conditioned malicious responses while preserving benign model behavior. These findings suggest that backdoor removal in LLMs can be formulated as a localized structural repair problem rather than only a broad behavioral alignment problem.

Related papers

• ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety [54.4092272526747]
  Multimodal Large Language Models (MLLMs) have achieved remarkable success in cross-modal understanding and generation, yet their deployment is threatened by critical safety vulnerabilities.<n>We propose ProjLens, an interpretability framework designed to demystify MLLMs backdoors.
  arXiv  Detail & Related papers  (2026-04-21T04:52:38Z)
• Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors [48.881343993730844]
  Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines.<n> adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present.<n>Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors.
  arXiv  Detail & Related papers  (2026-04-14T06:48:33Z)
• Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Reference [15.215130286922564]
  Backdoor attacks pose severe security threats to large language models (LLMs)<n>We propose a new framework for instruction-tuned LLM without prior trigger knowledge or clean references.<n>We produce purified models that withstand diverse backdoor attacks and threat models while preserving generative capability.
  arXiv  Detail & Related papers  (2026-03-13T17:09:37Z)
• Backdoor Unlearning by Linear Task Decomposition [69.91984435094157]
  Foundation models are highly susceptible to adversarial perturbations and targeted backdoor attacks.<n>Existing backdoor removal approaches rely on costly fine-tuning to override the harmful behavior.<n>This raises the question of whether backdoors can be removed without compromising the general capabilities of the models.
  arXiv  Detail & Related papers  (2025-10-16T16:18:07Z)
• Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution [49.78359632298156]
  Large language models (LLMs) have seen significant advancements, achieving superior performance in various Natural Language Processing (NLP) tasks.<n> backdoor attacks, where models behave normally for standard queries but generate harmful responses or unintended output when specific triggers are activated.<n>We present LETHE, a novel method to eliminate backdoor behaviors from LLMs through knowledge dilution.
  arXiv  Detail & Related papers  (2025-08-28T17:05:18Z)
• Robust Anti-Backdoor Instruction Tuning in LVLMs [53.766434746801366]
  We introduce a lightweight, certified-agnostic defense framework for large visual language models (LVLMs)<n>Our framework finetunes only adapter modules and text embedding layers under instruction tuning.<n>Experiments against seven attacks on Flickr30k and MSCOCO demonstrate that ours reduces their attack success rate to nearly zero.
  arXiv  Detail & Related papers  (2025-06-04T01:23:35Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning [12.293101110323722]
  Fine-tuning-as-a-service exposes models to harmful fine-tuning attacks.<n>We propose a paradigm shift: instead of selective removal, we advocate for inducing model collapse.<n>This collapse directly neutralizes the very general capabilities that attackers exploit.
  arXiv  Detail & Related papers  (2025-05-22T11:47:08Z)
• ASPIRER: Bypassing System Prompts With Permutation-based Backdoors in LLMs [17.853862145962292]
  We introduce a novel backdoor attack that systematically bypasses system prompts. Our method achieves an attack success rate (ASR) of up to 99.50% while maintaining a clean accuracy (CACC) of 98.58%.
  arXiv  Detail & Related papers  (2024-10-05T02:58:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.