Prompting the Unseen: Detecting Hidden Backdoors in Black-Box Models

• URL: http://arxiv.org/abs/2411.09540v2
• Date: Mon, 07 Apr 2025 08:55:40 GMT
• Title: Prompting the Unseen: Detecting Hidden Backdoors in Black-Box Models
• Abstract summary: Visual prompting (VP) adapts well-trained frozen models for source domain tasks to target domain tasks.<n>textscBProm is a black-box model-level detection method to identify backdoors in suspicious models.
• Score: 26.918410899715735
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual prompting (VP) is a new technique that adapts well-trained frozen models for source domain tasks to target domain tasks. This study examines VP's benefits for black-box model-level backdoor detection. The visual prompt in VP maps class subspaces between source and target domains. We identify a misalignment, termed class subspace inconsistency, between clean and poisoned datasets. Based on this, we introduce \textsc{BProm}, a black-box model-level detection method to identify backdoors in suspicious models, if any. \textsc{BProm} leverages the low classification accuracy of prompted models when backdoors are present. Extensive experiments confirm \textsc{BProm}'s effectiveness.

Related papers

• EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy [66.59724477993339]
  Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across various tasks, yet they remain vulnerable to backdoor attacks.<n>We propose EntropyScan, a lightweight and trigger-agnostic method for model-level backdoor detection in LVLMs.<n>Our code will be publicly available soon.
  arXiv  Detail & Related papers  (2026-05-15T08:01:32Z)
• BlackMirror: Black-Box Backdoor Detection for Text-to-Image Models via Instruction-Response Deviation [117.54208768824869]
  This paper investigates the challenging task of detecting backdoored text-to-image models under black-box settings.<n>It introduces a novel detection framework BlackMirror.
  arXiv  Detail & Related papers  (2026-03-06T05:22:46Z)
• Detecting Semantic Backdoors in a Mystery Shopping Scenario [0.0]
  We tackle the problem of detecting semantic backdoors in classification models.<n>Under the assumption that the clean training dataset and the training recipe of the model are both known, we propose a reference model pool.<n>We experimentally analyze a number of approaches to compute model distances and we also test a scenario where the provider performs an adaptive attack to avoid detection.
  arXiv  Detail & Related papers  (2026-01-07T11:04:04Z)
• BURN: Backdoor Unlearning via Adversarial Boundary Analysis [73.14147934175604]
  Backdoor unlearning aims to remove backdoor-related information while preserving the model's original functionality.<n>We propose Backdoor Unlearning via adversaRial bouNdary analysis (BURN), a novel defense framework that integrates false correlation decoupling, progressive data refinement, and model purification.
  arXiv  Detail & Related papers  (2025-07-14T17:13:06Z)
• Proactive Adversarial Defense: Harnessing Prompt Tuning in Vision-Language Models to Detect Unseen Backdoored Images [0.0]
  Backdoor attacks pose a critical threat by embedding hidden triggers into inputs, causing models to misclassify them into target labels. We introduce a groundbreaking method to detect unseen backdoored images during both training and inference. Our approach trains learnable text prompts to differentiate clean images from those with hidden backdoor triggers.
  arXiv  Detail & Related papers  (2024-12-11T19:54:14Z)
• DREAM: Domain-agnostic Reverse Engineering Attributes of Black-box Model [50.94236887900527]
  We present a new problem of black-box reverse engineering, without requiring the availability of the target model's training dataset.<n>We learn a domain-agnostic meta-model to infer the attributes of the target black-box model with unknown training data.
  arXiv  Detail & Related papers  (2024-12-08T07:37:05Z)
• Backdoor Defense through Self-Supervised and Generative Learning [0.0]
  Training on such data injects a backdoor which causes malicious inference in selected test samples. This paper explores an approach based on generative modelling of per-class distributions in a self-supervised representation space. In both cases, we find that per-class generative models allow to detect poisoned data and cleanse the dataset.
  arXiv  Detail & Related papers  (2024-09-02T11:40:01Z)
• UFID: A Unified Framework for Input-level Backdoor Detection on Diffusion Models [19.46962670935554]
  Diffusion Models are vulnerable to backdoor attacks. malicious attackers inject backdoors by poisoning some parts of the training samples. This poses a serious threat to the downstream users, who query the diffusion models through the API or directly download them from the internet.
  arXiv  Detail & Related papers  (2024-04-01T13:21:05Z)
• Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks [63.269788236474234]
  We propose to use model pairs on open-set classification tasks for detecting backdoors. We show that this score, can be an indicator for the presence of a backdoor despite models being of different architectures. This technique allows for the detection of backdoors on models designed for open-set classification tasks, which is little studied in the literature.
  arXiv  Detail & Related papers  (2024-02-28T21:29:16Z)
• XGBD: Explanation-Guided Graph Backdoor Detection [21.918945251903523]
  Backdoor attacks pose a significant security risk to graph learning models. We propose an explanation-guided backdoor detection method to take advantage of the topological information.
  arXiv  Detail & Related papers  (2023-08-08T17:10:23Z)
• DREAM: Domain-free Reverse Engineering Attributes of Black-box Model [51.37041886352823]
  We propose a new problem of Domain-agnostic Reverse Engineering the Attributes of a black-box target model. We learn a domain-agnostic model to infer the attributes of a target black-box model with unknown training data.
  arXiv  Detail & Related papers  (2023-07-20T16:25:58Z)
• Backdoor Learning on Sequence to Sequence Models [94.23904400441957]
  In this paper, we study whether sequence-to-sequence (seq2seq) models are vulnerable to backdoor attacks. Specifically, we find by only injecting 0.2% samples of the dataset, we can cause the seq2seq model to generate the designated keyword and even the whole sentence. Extensive experiments on machine translation and text summarization have been conducted to show our proposed methods could achieve over 90% attack success rate on multiple datasets and models.
  arXiv  Detail & Related papers  (2023-05-03T20:31:13Z)
• Mask and Restore: Blind Backdoor Defense at Test Time with Masked Autoencoder [57.739693628523]
  We propose a framework for blind backdoor defense with Masked AutoEncoder (BDMAE) BDMAE detects possible triggers in the token space using image structural similarity and label consistency between the test image and MAE restorations. Our approach is blind to the model restorations, trigger patterns and image benignity.
  arXiv  Detail & Related papers  (2023-03-27T19:23:33Z)
• Training set cleansing of backdoor poisoning by self-supervised representation learning [0.0]
  A backdoor or Trojan attack is an important type of data poisoning attack against deep neural network (DNN) We show that supervised training may build stronger association between the backdoor pattern and the associated target class than that between normal features and the true class of origin. We propose to use unsupervised representation learning to avoid emphasising backdoor-poisoned training samples and learn a similar feature embedding for samples of the same class.
  arXiv  Detail & Related papers  (2022-10-19T03:29:58Z)
• Black-box Detection of Backdoor Attacks with Limited Information and Data [56.0735480850555]
  We propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models.
  arXiv  Detail & Related papers  (2021-03-24T12:06:40Z)
• Scalable Backdoor Detection in Neural Networks [61.39635364047679]
  Deep learning models are vulnerable to Trojan attacks, where an attacker can install a backdoor during training time to make the resultant model misidentify samples contaminated with a small trigger patch. We propose a novel trigger reverse-engineering based approach whose computational complexity does not scale with the number of labels, and is based on a measure that is both interpretable and universal across different network and patch types. In experiments, we observe that our method achieves a perfect score in separating Trojaned models from pure models, which is an improvement over the current state-of-the art method.
  arXiv  Detail & Related papers  (2020-06-10T04:12:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.