Uncovering and Aligning Anomalous Attention Heads to Defend Against NLP Backdoor Attacks

• URL: http://arxiv.org/abs/2511.13789v1
• Date: Sun, 16 Nov 2025 15:26:50 GMT
• Title: Uncovering and Aligning Anomalous Attention Heads to Defend Against NLP Backdoor Attacks
• Authors: Haotian Jin, Yang Li, Haihui Fan, Lin Shen, Xiangfang Li, Bo Li,
• Abstract summary: Backdoor attacks pose a serious threat to the security of large language models.<n>We propose a backdoor detection method based on attention similarity.<n>Our method significantly reduces the success rate of backdoor attacks.
• Score: 9.078969469946038
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Backdoor attacks pose a serious threat to the security of large language models (LLMs), causing them to exhibit anomalous behavior under specific trigger conditions. The design of backdoor triggers has evolved from fixed triggers to dynamic or implicit triggers. This increased flexibility in trigger design makes it challenging for defenders to identify their specific forms accurately. Most existing backdoor defense methods are limited to specific types of triggers or rely on an additional clean model for support. To address this issue, we propose a backdoor detection method based on attention similarity, enabling backdoor detection without prior knowledge of the trigger. Our study reveals that models subjected to backdoor attacks exhibit unusually high similarity among attention heads when exposed to triggers. Based on this observation, we propose an attention safety alignment approach combined with head-wise fine-tuning to rectify potentially contaminated attention heads, thereby effectively mitigating the impact of backdoor attacks. Extensive experimental results demonstrate that our method significantly reduces the success rate of backdoor attacks while preserving the model's performance on downstream tasks.

Related papers

• Towards Effective, Stealthy, and Persistent Backdoor Attacks Targeting Graph Foundation Models [62.87838888016534]
  Graph Foundation Models (GFMs) are pre-trained on diverse source domains and adapted to unseen targets.<n>Backdoor attacks against GFMs are non-trivial due to three key challenges.<n>We propose GFM-BA, a novel Backdoor Attack model against Graph Foundation Models.
  arXiv  Detail & Related papers  (2025-11-22T08:52:09Z)
• Backdoor Unlearning by Linear Task Decomposition [69.91984435094157]
  Foundation models are highly susceptible to adversarial perturbations and targeted backdoor attacks.<n>Existing backdoor removal approaches rely on costly fine-tuning to override the harmful behavior.<n>This raises the question of whether backdoors can be removed without compromising the general capabilities of the models.
  arXiv  Detail & Related papers  (2025-10-16T16:18:07Z)
• Trigger without Trace: Towards Stealthy Backdoor Attack on Text-to-Image Diffusion Models [70.03122709795122]
  Backdoor attacks targeting text-to-image diffusion models have advanced rapidly.<n>Current backdoor samples often exhibit two key abnormalities compared to benign samples.<n>We propose Trigger without Trace (TwT) by explicitly mitigating these consistencies.
  arXiv  Detail & Related papers  (2025-03-22T10:41:46Z)
• A4O: All Trigger for One sample [10.78460062665304]
  We show that proposed backdoor defenders often rely on the assumption that triggers would appear in a unified way.<n>In this paper, we show that this naive assumption can create a loophole, allowing more sophisticated backdoor attacks to bypass.<n>We design a novel backdoor attack mechanism that incorporates multiple types of backdoor triggers, focusing on stealthiness and effectiveness.
  arXiv  Detail & Related papers  (2025-01-13T10:38:58Z)
• Backdoor Attack with Invisible Triggers Based on Model Architecture Modification [5.094386595197844]
  Traditional backdoor attacks involve injecting malicious samples with specific triggers into the training data.<n>More sophisticated attacks modify the model's architecture directly.<n>A novel backdoor attack method is presented in the paper.<n>It embeds the backdoor within the model's architecture and has the capability to generate inconspicuous and stealthy triggers.
  arXiv  Detail & Related papers  (2024-12-22T07:39:43Z)
• Neutralizing Backdoors through Information Conflicts for Large Language Models [20.6331157117675]
  We present a novel method to eliminate backdoor behaviors from large language models (LLMs)<n>We leverage a lightweight dataset to train a conflict model, which is then merged with the backdoored model to neutralize malicious behaviors.<n>We can reduce the attack success rate of advanced backdoor attacks by up to 98% while maintaining over 90% clean data accuracy.
  arXiv  Detail & Related papers  (2024-11-27T12:15:22Z)
• BEEAR: Embedding-based Adversarial Removal of Safety Backdoors in Instruction-tuned Language Models [57.5404308854535]
  Safety backdoor attacks in large language models (LLMs) enable the stealthy triggering of unsafe behaviors while evading detection during normal interactions. We present BEEAR, a mitigation approach leveraging the insight that backdoor triggers induce relatively uniform drifts in the model's embedding space. Our bi-level optimization method identifies universal embedding perturbations that elicit unwanted behaviors and adjusts the model parameters to reinforce safe behaviors against these perturbations.
  arXiv  Detail & Related papers  (2024-06-24T19:29:47Z)
• LOTUS: Evasive and Resilient Backdoor Attacks through Sub-Partitioning [49.174341192722615]
  Backdoor attack poses a significant security threat to Deep Learning applications. Recent papers have introduced attacks using sample-specific invisible triggers crafted through special transformation functions. We introduce a novel backdoor attack LOTUS to address both evasiveness and resilience.
  arXiv  Detail & Related papers  (2024-03-25T21:01:29Z)
• Demystifying Poisoning Backdoor Attacks from a Statistical Perspective [35.30533879618651]
  Backdoor attacks pose a significant security risk due to their stealthy nature and potentially serious consequences. This paper evaluates the effectiveness of any backdoor attack incorporating a constant trigger. Our derived understanding applies to both discriminative and generative models.
  arXiv  Detail & Related papers  (2023-10-16T19:35:01Z)
• Untargeted Backdoor Attack against Object Detection [69.63097724439886]
  We design a poison-only backdoor attack in an untargeted manner, based on task characteristics. We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns.
  arXiv  Detail & Related papers  (2022-11-02T17:05:45Z)
• Rethinking the Backdoor Attacks' Triggers: A Frequency Perspective [10.03897682559064]
  This paper revisits existing backdoor triggers from a frequency perspective and performs a comprehensive analysis. We show that many current backdoor attacks exhibit severe high-frequency artifacts, which persist across different datasets and resolutions. We propose a practical way to create smooth backdoor triggers without high-frequency artifacts and study their detectability.
  arXiv  Detail & Related papers  (2021-04-07T22:05:28Z)
• Rethinking the Trigger of Backdoor Attack [83.98031510668619]
  Currently, most of existing backdoor attacks adopted the setting of emphstatic trigger, $i.e.,$ triggers across the training and testing images follow the same appearance and are located in the same area. We demonstrate that such an attack paradigm is vulnerable when the trigger in testing images is not consistent with the one used for training.
  arXiv  Detail & Related papers  (2020-04-09T17:19:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.