Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks

• URL: http://arxiv.org/abs/2402.18718v2
• Date: Mon, 04 Nov 2024 13:40:24 GMT
• Title: Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks
• Authors: Alexander Unnervik, Hatef Otroshi Shahreza, Anjith George, Sébastien Marcel,
• Abstract summary: We propose to use model pairs on open-set classification tasks for detecting backdoors. We show that this score, can be an indicator for the presence of a backdoor despite models being of different architectures. This technique allows for the detection of backdoors on models designed for open-set classification tasks, which is little studied in the literature.
• Score: 63.269788236474234
• License:
• Abstract: Backdoor attacks allow an attacker to embed a specific vulnerability in a machine learning algorithm, activated when an attacker-chosen pattern is presented, causing a specific misprediction. The need to identify backdoors in biometric scenarios has led us to propose a novel technique with different trade-offs. In this paper we propose to use model pairs on open-set classification tasks for detecting backdoors. Using a simple linear operation to project embeddings from a probe model's embedding space to a reference model's embedding space, we can compare both embeddings and compute a similarity score. We show that this score, can be an indicator for the presence of a backdoor despite models being of different architectures, having been trained independently and on different datasets. This technique allows for the detection of backdoors on models designed for open-set classification tasks, which is little studied in the literature. Additionally, we show that backdoors can be detected even when both models are backdoored. The source code is made available for reproducibility purposes.

Related papers

• Exploiting the Vulnerability of Large Language Models via Defense-Aware Architectural Backdoor [0.24335447922683692]
  We introduce a new type of backdoor attack that conceals itself within the underlying model architecture. The add-on modules of model architecture layers can detect the presence of input trigger tokens and modify layer weights. We conduct extensive experiments to evaluate our attack methods using two model architecture settings on five different large language datasets.
  arXiv  Detail & Related papers  (2024-09-03T14:54:16Z)
• Backdoor Learning on Sequence to Sequence Models [94.23904400441957]
  In this paper, we study whether sequence-to-sequence (seq2seq) models are vulnerable to backdoor attacks. Specifically, we find by only injecting 0.2% samples of the dataset, we can cause the seq2seq model to generate the designated keyword and even the whole sentence. Extensive experiments on machine translation and text summarization have been conducted to show our proposed methods could achieve over 90% attack success rate on multiple datasets and models.
  arXiv  Detail & Related papers  (2023-05-03T20:31:13Z)
• Detecting Backdoors in Deep Text Classifiers [43.36440869257781]
  We present the first robust defence mechanism that generalizes to several backdoor attacks against text classification models. Our technique is highly accurate at defending against state-of-the-art backdoor attacks, including data poisoning and weight poisoning.
  arXiv  Detail & Related papers  (2022-10-11T07:48:03Z)
• An anomaly detection approach for backdoored neural networks: face recognition as a case study [77.92020418343022]
  We propose a novel backdoored network detection method based on the principle of anomaly detection. We test our method on a novel dataset of backdoored networks and report detectability results with perfect scores.
  arXiv  Detail & Related papers  (2022-08-22T12:14:13Z)
• Architectural Backdoors in Neural Networks [27.315196801989032]
  We introduce a new class of backdoor attacks that hide inside model architectures. These backdoors are simple to implement, for instance by publishing open-source code for a backdoored model architecture. We demonstrate that model architectural backdoors represent a real threat and, unlike other approaches, can survive a complete re-training from scratch.
  arXiv  Detail & Related papers  (2022-06-15T22:44:03Z)
• Planting Undetectable Backdoors in Machine Learning Models [17.494133972292403]
  We show how a malicious learner can plant an undetectable backdoor into a classifier. Without the appropriate "backdoor key", the mechanism is hidden and cannot be detected by any computationally-bounded observer. We show two frameworks for planting undetectable backdoors, with incomparable guarantees.
  arXiv  Detail & Related papers  (2022-04-14T13:55:21Z)
• Check Your Other Door! Establishing Backdoor Attacks in the Frequency Domain [80.24811082454367]
  We show the advantages of utilizing the frequency domain for establishing undetectable and powerful backdoor attacks. We also show two possible defences that succeed against frequency-based backdoor attacks and possible ways for the attacker to bypass them.
  arXiv  Detail & Related papers  (2021-09-12T12:44:52Z)
• Black-box Detection of Backdoor Attacks with Limited Information and Data [56.0735480850555]
  We propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models.
  arXiv  Detail & Related papers  (2021-03-24T12:06:40Z)
• Scalable Backdoor Detection in Neural Networks [61.39635364047679]
  Deep learning models are vulnerable to Trojan attacks, where an attacker can install a backdoor during training time to make the resultant model misidentify samples contaminated with a small trigger patch. We propose a novel trigger reverse-engineering based approach whose computational complexity does not scale with the number of labels, and is based on a measure that is both interpretable and universal across different network and patch types. In experiments, we observe that our method achieves a perfect score in separating Trojaned models from pure models, which is an improvement over the current state-of-the art method.
  arXiv  Detail & Related papers  (2020-06-10T04:12:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.