Backdoor Directions in Vision Transformers

• URL: http://arxiv.org/abs/2603.10806v1
• Date: Wed, 11 Mar 2026 14:13:48 GMT
• Title: Backdoor Directions in Vision Transformers
• Abstract summary: This paper investigates how Backdoor Attacks are represented within Vision Transformers (ViTs)<n>We identify a specific trigger direction'' in the model's activations that corresponds to the internal representation of the trigger.<n>Using this direction as a diagnostic tool, we trace how backdoor features are processed across layers.
• Score: 56.382912038371046
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper investigates how Backdoor Attacks are represented within Vision Transformers (ViTs). By assuming knowledge of the trigger, we identify a specific ``trigger direction'' in the model's activations that corresponds to the internal representation of the trigger. We confirm the causal role of this linear direction by showing that interventions in both activation and parameter space consistently modulate the model's backdoor behavior across multiple datasets and attack types. Using this direction as a diagnostic tool, we trace how backdoor features are processed across layers. Our analysis reveals distinct qualitative differences: static-patch triggers follow a different internal logic than stealthy, distributed triggers. We further examine the link between backdoors and adversarial attacks, specifically testing whether PGD-based perturbations (de-)activate the identified trigger mechanism. Finally, we propose a data-free, weight-based detection scheme for stealthy-trigger attacks. Our findings show that mechanistic interpretability offers a robust framework for diagnosing and addressing security vulnerabilities in computer vision.

Related papers

• BehaviorGuard: Online Backdoor Defense for Deep Reinforcement Learning [32.53874142863277]
  Backdoor attacks pose a serious threat to deep reinforcement learning (DRL)<n>Current defenses rely on reward anomalies to reverse-engineer triggers and model finetuning to remove backdoors.<n>We propose BehaviorGuard, an online behavior-based backdoor detection and mitigation framework for DRL.
  arXiv  Detail & Related papers  (2026-05-07T10:21:14Z)
• Uncovering and Aligning Anomalous Attention Heads to Defend Against NLP Backdoor Attacks [9.078969469946038]
  Backdoor attacks pose a serious threat to the security of large language models.<n>We propose a backdoor detection method based on attention similarity.<n>Our method significantly reduces the success rate of backdoor attacks.
  arXiv  Detail & Related papers  (2025-11-16T15:26:50Z)
• Trigger without Trace: Towards Stealthy Backdoor Attack on Text-to-Image Diffusion Models [70.03122709795122]
  Backdoor attacks targeting text-to-image diffusion models have advanced rapidly.<n>Current backdoor samples often exhibit two key abnormalities compared to benign samples.<n>We propose Trigger without Trace (TwT) by explicitly mitigating these consistencies.
  arXiv  Detail & Related papers  (2025-03-22T10:41:46Z)
• Backdoor Attack with Invisible Triggers Based on Model Architecture Modification [5.094386595197844]
  Traditional backdoor attacks involve injecting malicious samples with specific triggers into the training data.<n>More sophisticated attacks modify the model's architecture directly.<n>A novel backdoor attack method is presented in the paper.<n>It embeds the backdoor within the model's architecture and has the capability to generate inconspicuous and stealthy triggers.
  arXiv  Detail & Related papers  (2024-12-22T07:39:43Z)
• Proactive Adversarial Defense: Harnessing Prompt Tuning in Vision-Language Models to Detect Unseen Backdoored Images [0.0]
  Backdoor attacks pose a critical threat by embedding hidden triggers into inputs, causing models to misclassify them into target labels.<n>We introduce a groundbreaking method to detect unseen backdoored images during both training and inference.<n>Our approach trains learnable text prompts to differentiate clean images from those with hidden backdoor triggers.
  arXiv  Detail & Related papers  (2024-12-11T19:54:14Z)
• Long-Tailed Backdoor Attack Using Dynamic Data Augmentation Operations [50.1394620328318]
  Existing backdoor attacks mainly focus on balanced datasets. We propose an effective backdoor attack named Dynamic Data Augmentation Operation (D$2$AO) Our method can achieve the state-of-the-art attack performance while preserving the clean accuracy.
  arXiv  Detail & Related papers  (2024-10-16T18:44:22Z)
• Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks [63.269788236474234]
  We propose to use model pairs on open-set classification tasks for detecting backdoors. We show that this score, can be an indicator for the presence of a backdoor despite models being of different architectures. This technique allows for the detection of backdoors on models designed for open-set classification tasks, which is little studied in the literature.
  arXiv  Detail & Related papers  (2024-02-28T21:29:16Z)
• Untargeted Backdoor Attack against Object Detection [69.63097724439886]
  We design a poison-only backdoor attack in an untargeted manner, based on task characteristics. We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns.
  arXiv  Detail & Related papers  (2022-11-02T17:05:45Z)
• Backdoor Learning Curves: Explaining Backdoor Poisoning Beyond Influence Functions [23.750285504961337]
  We study the process of backdoor learning under the lens of incremental learning and influence functions.<n>We show that the effectiveness of backdoor attacks depends on: (i) the complexity of the learning algorithm; (ii) the fraction of backdoor samples injected into the training set; and (iii) the size and visibility of the backdoor trigger.
  arXiv  Detail & Related papers  (2021-06-14T08:00:48Z)
• Rethinking the Trigger of Backdoor Attack [83.98031510668619]
  Currently, most of existing backdoor attacks adopted the setting of emphstatic trigger, $i.e.,$ triggers across the training and testing images follow the same appearance and are located in the same area. We demonstrate that such an attack paradigm is vulnerable when the trigger in testing images is not consistent with the one used for training.
  arXiv  Detail & Related papers  (2020-04-09T17:19:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.