DEFUSE: Generalizable Backdoor Defense for Self-Supervised Encoders with Generative Priors

• URL: http://arxiv.org/abs/2608.25851v1
• Date: Wed, 26 Aug 2026 14:24:05 GMT
• Title: DEFUSE: Generalizable Backdoor Defense for Self-Supervised Encoders with Generative Priors
• Abstract summary: Self-supervised learning (SSL) encoders are vulnerable to backdoor attacks.<n>DEFUSE is a generalizable backdoor detection framework for SSL encoders.
• Score: 40.98521902229488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-supervised learning (SSL) encoders are vulnerable to backdoor attacks, posing threats to both visual SSL encoders and vision-language encoders. Existing defenses are typically designed for only one of these paradigms and rely on restrictive assumptions such as access to uninfected in-distribution data or precomputed pseudo-labels, which are difficult to satisfy in practice. To address these limitations, we propose DEFUSE, a generalizable backdoor detection framework for SSL encoders. Inspired by Bayesian posterior inference, we reformulate backdoor detection as a representation-conditioned image likelihood estimation problem parameterized by a conditional diffusion generative model. Uninfected representations tend to yield semantically consistent reconstructions, whereas backdoored ones are more likely to be mapped to the attacker's target class or semantically meaningless images, deviating from the original semantics and thereby exposing the backdoor. However, we find that the exact likelihood is intractable, because highly abstracted representations discard the low-level information necessary for pixel-faithful reconstruction. We therefore relax the objective to semantic reconstruction and evaluate it in a well-separated representation space provided by a reference encoder. Rather than training from scratch, we fine-tune a pretrained diffusion model, leveraging its generative prior to map data onto the natural image manifold while preserving semantic content. Extensive experiments demonstrate that DEFUSE substantially outperforms existing detectors across diverse attack settings, generalizing to both visual SSL and vision-language encoders. Notably, our method greatly reduces the reliance on prior knowledge about the victim encoder or the attack strategy. The source code is available at https://github.com/jsrdcht/DEFUSE .

Related papers

• Rethinking Security in Semantic Communication: Latent Manipulation as a New Threat [4.488447044579913]
  Deep learning-based semantic communication (SemCom) has emerged as a promising paradigm for next-generation wireless networks.<n>In this paper, we uncover a fundamental latent-space vulnerability that enables Man-in-the-Middle (MitM) attacker to covertly manipulate the transmitted semantics.
  arXiv  Detail & Related papers  (2025-12-03T01:54:11Z)
• Backdoor Attack on Vision Language Models with Stealthy Semantic Manipulation [32.24294112337828]
  BadSem is a data poisoning attack that injects backdoors by deliberately misaligning image-text pairs during training.<n>Our experiments show that BadSem achieves over 98% average ASR, generalizes well to out-of-distribution datasets, and can transfer across poisoning modalities.<n>Our findings highlight the urgent need to address semantic vulnerabilities in Vision Language Models for their safer deployment.
  arXiv  Detail & Related papers  (2025-06-08T16:40:40Z)
• Robust Anti-Backdoor Instruction Tuning in LVLMs [53.766434746801366]
  We introduce a lightweight, certified-agnostic defense framework for large visual language models (LVLMs)<n>Our framework finetunes only adapter modules and text embedding layers under instruction tuning.<n>Experiments against seven attacks on Flickr30k and MSCOCO demonstrate that ours reduces their attack success rate to nearly zero.
  arXiv  Detail & Related papers  (2025-06-04T01:23:35Z)
• Propaganda via AI? A Study on Semantic Backdoors in Large Language Models [7.282200564983221]
  We show that semantic backdoors can be implanted with only a small poisoned corpus.<n>We introduce a black-box detection framework, RAVEN, which combines semantic entropy with cross-model consistency analysis.<n> Empirical evaluations uncover previously undetected semantic backdoors.
  arXiv  Detail & Related papers  (2025-04-15T16:43:15Z)
• Reformulation is All You Need: Addressing Malicious Text Features in DNNs [53.45564571192014]
  We propose a unified and adaptive defense framework that is effective against both adversarial and backdoor attacks.<n>Our framework outperforms existing sample-oriented defense baselines across a diverse range of malicious textual features.
  arXiv  Detail & Related papers  (2025-02-02T03:39:43Z)
• DeDe: Detecting Backdoor Samples for SSL Encoders via Decoders [6.698677477097004]
  Self-supervised learning (SSL) is pervasively exploited in training high-quality upstream encoders with a large amount of unlabeled data.<n>Victim encoders associate triggered inputs with target embeddings, such that the downstream tasks inherit unintended behaviors when the trigger is activated.<n>We propose a novel detection mechanism, DeDe, which detects the activation of backdoor mappings caused by triggered inputs on victim encoders.
  arXiv  Detail & Related papers  (2024-11-25T07:26:22Z)
• EmInspector: Combating Backdoor Attacks in Federated Self-Supervised Learning Through Embedding Inspection [53.25863925815954]
  Federated self-supervised learning (FSSL) has emerged as a promising paradigm that enables the exploitation of clients' vast amounts of unlabeled data. While FSSL offers advantages, its susceptibility to backdoor attacks has not been investigated. We propose the Embedding Inspector (EmInspector) that detects malicious clients by inspecting the embedding space of local models.
  arXiv  Detail & Related papers  (2024-05-21T06:14:49Z)
• Downstream-agnostic Adversarial Examples [66.8606539786026]
  AdvEncoder is first framework for generating downstream-agnostic universal adversarial examples based on pre-trained encoder. Unlike traditional adversarial example works, the pre-trained encoder only outputs feature vectors rather than classification labels. Our results show that an attacker can successfully attack downstream tasks without knowing either the pre-training dataset or the downstream dataset.
  arXiv  Detail & Related papers  (2023-07-23T10:16:47Z)
• Hidden Backdoor Attack against Semantic Segmentation Models [60.0327238844584]
  The emphbackdoor attack intends to embed hidden backdoors in deep neural networks (DNNs) by poisoning training data. We propose a novel attack paradigm, the emphfine-grained attack, where we treat the target label from the object-level instead of the image-level. Experiments show that the proposed methods can successfully attack semantic segmentation models by poisoning only a small proportion of training data.
  arXiv  Detail & Related papers  (2021-03-06T05:50:29Z)
• A Neuro-Inspired Autoencoding Defense Against Adversarial Perturbations [11.334887948796611]
  Deep Neural Networks (DNNs) are vulnerable to adversarial attacks. Most effective current defense is to train the network using adversarially perturbed examples. In this paper, we investigate a radically different, neuro-inspired defense mechanism.
  arXiv  Detail & Related papers  (2020-11-21T21:03:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.