Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation

• URL: http://arxiv.org/abs/2505.18323v1
• Date: Fri, 23 May 2025 19:28:45 GMT
• Title: Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation
• Authors: Nicolas Küchler, Ivan Petrov, Conrad Grobler, Ilia Shumailov,
• Abstract summary: We introduce a novel class of backdoors that builds upon recent advancements in architectural backdoors.<n>We show that such attacks are not only feasible but also alarmingly effective.<n>We propose a deterministic mitigation strategy that provides formal guarantees against this new attack vector.
• Score: 9.961238260113916
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: For nearly a decade the academic community has investigated backdoors in neural networks, primarily focusing on classification tasks where adversaries manipulate the model prediction. While demonstrably malicious, the immediate real-world impact of such prediction-altering attacks has remained unclear. In this paper we introduce a novel and significantly more potent class of backdoors that builds upon recent advancements in architectural backdoors. We demonstrate how these backdoors can be specifically engineered to exploit batched inference, a common technique for hardware utilization, enabling large-scale user data manipulation and theft. By targeting the batching process, these architectural backdoors facilitate information leakage between concurrent user requests and allow attackers to fully control model responses directed at other users within the same batch. In other words, an attacker who can change the model architecture can set and steal model inputs and outputs of other users within the same batch. We show that such attacks are not only feasible but also alarmingly effective, can be readily injected into prevalent model architectures, and represent a truly malicious threat to user privacy and system integrity. Critically, to counteract this new class of vulnerabilities, we propose a deterministic mitigation strategy that provides formal guarantees against this new attack vector, unlike prior work that relied on Large Language Models to find the backdoors. Our mitigation strategy employs a novel Information Flow Control mechanism that analyzes the model graph and proves non-interference between different user inputs within the same batch. Using our mitigation strategy we perform a large scale analysis of models hosted through Hugging Face and find over 200 models that introduce (unintended) information leakage between batch entries due to the use of dynamic quantization.

Related papers

• Behavior Backdoor for Deep Learning Models [95.50787731231063]
  We take the first step towards behavioral backdoor'' attack, which is defined as a behavior-triggered backdoor model training procedure.<n>We propose the first pipeline of implementing behavior backdoor, i.e., the Quantification Backdoor (QB) attack.<n>Experiments have been conducted on different models, datasets, and tasks, demonstrating the effectiveness of this novel backdoor attack.
  arXiv  Detail & Related papers  (2024-12-02T10:54:02Z)
• Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models [112.48136829374741]
  In this paper, we unveil a new vulnerability: the privacy backdoor attack. When a victim fine-tunes a backdoored model, their training data will be leaked at a significantly higher rate than if they had fine-tuned a typical model. Our findings highlight a critical privacy concern within the machine learning community and call for a reevaluation of safety protocols in the use of open-source pre-trained models.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks [63.269788236474234]
  We propose to use model pairs on open-set classification tasks for detecting backdoors. We show that this score, can be an indicator for the presence of a backdoor despite models being of different architectures. This technique allows for the detection of backdoors on models designed for open-set classification tasks, which is little studied in the literature.
  arXiv  Detail & Related papers  (2024-02-28T21:29:16Z)
• Hiding Backdoors within Event Sequence Data via Poisoning Attacks [2.532893215351299]
  In computer vision, one can shape the output during inference by performing an adversarial attack called poisoning. For sequences of financial transactions of a customer, insertion of a backdoor is harder to perform. We replace a clean model with a poisoned one that is aware of the availability of a backdoor and utilize this knowledge.
  arXiv  Detail & Related papers  (2023-08-20T08:27:42Z)
• Architectural Backdoors in Neural Networks [27.315196801989032]
  We introduce a new class of backdoor attacks that hide inside model architectures. These backdoors are simple to implement, for instance by publishing open-source code for a backdoored model architecture. We demonstrate that model architectural backdoors represent a real threat and, unlike other approaches, can survive a complete re-training from scratch.
  arXiv  Detail & Related papers  (2022-06-15T22:44:03Z)
• On the Effectiveness of Adversarial Training against Backdoor Attacks [111.8963365326168]
  A backdoored model always predicts a target class in the presence of a predefined trigger pattern. In general, adversarial training is believed to defend against backdoor attacks. We propose a hybrid strategy which provides satisfactory robustness across different backdoor attacks.
  arXiv  Detail & Related papers  (2022-02-22T02:24:46Z)
• DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection [26.593268413299228]
  Federated Learning (FL) allows multiple clients to collaboratively train a Neural Network (NN) model on their private data without revealing the data. DeepSight is a novel model filtering approach for mitigating backdoor attacks. We show that it can mitigate state-of-the-art backdoor attacks with a negligible impact on the model's performance on benign data.
  arXiv  Detail & Related papers  (2022-01-03T17:10:07Z)
• Check Your Other Door! Establishing Backdoor Attacks in the Frequency Domain [80.24811082454367]
  We show the advantages of utilizing the frequency domain for establishing undetectable and powerful backdoor attacks. We also show two possible defences that succeed against frequency-based backdoor attacks and possible ways for the attacker to bypass them.
  arXiv  Detail & Related papers  (2021-09-12T12:44:52Z)
• Black-box Detection of Backdoor Attacks with Limited Information and Data [56.0735480850555]
  We propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models.
  arXiv  Detail & Related papers  (2021-03-24T12:06:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.