Backdoor Detection through Replicated Execution of Outsourced Training

• URL: http://arxiv.org/abs/2504.00170v1
• Date: Mon, 31 Mar 2025 19:26:34 GMT
• Title: Backdoor Detection through Replicated Execution of Outsourced Training
• Abstract summary: We show that a client with access to multiple cloud providers can replicate a subset of training steps across multiple servers to detect deviation from the training procedure.<n>We identify malicious servers by the substantial difference between model updates required for backdooring and those resulting from clean training.
• Score: 34.202306551388396
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It is common practice to outsource the training of machine learning models to cloud providers. Clients who do so gain from the cloud's economies of scale, but implicitly assume trust: the server should not deviate from the client's training procedure. A malicious server may, for instance, seek to insert backdoors in the model. Detecting a backdoored model without prior knowledge of both the backdoor attack and its accompanying trigger remains a challenging problem. In this paper, we show that a client with access to multiple cloud providers can replicate a subset of training steps across multiple servers to detect deviation from the training procedure in a similar manner to differential testing. Assuming some cloud-provided servers are benign, we identify malicious servers by the substantial difference between model updates required for backdooring and those resulting from clean training. Perhaps the strongest advantage of our approach is its suitability to clients that have limited-to-no local compute capability to perform training; we leverage the existence of multiple cloud providers to identify malicious updates without expensive human labeling or heavy computation. We demonstrate the capabilities of our approach on an outsourced supervised learning task where $50\%$ of the cloud providers insert their own backdoor; our approach is able to correctly identify $99.6\%$ of them. In essence, our approach is successful because it replaces the signature-based paradigm taken by existing approaches with an anomaly-based detection paradigm. Furthermore, our approach is robust to several attacks from adaptive adversaries utilizing knowledge of our detection scheme.

Related papers

• Detecting Semantic Backdoors in a Mystery Shopping Scenario [0.0]
  We tackle the problem of detecting semantic backdoors in classification models.<n>Under the assumption that the clean training dataset and the training recipe of the model are both known, we propose a reference model pool.<n>We experimentally analyze a number of approaches to compute model distances and we also test a scenario where the provider performs an adaptive attack to avoid detection.
  arXiv  Detail & Related papers  (2026-01-07T11:04:04Z)
• Backdoor Unlearning by Linear Task Decomposition [69.91984435094157]
  Foundation models are highly susceptible to adversarial perturbations and targeted backdoor attacks.<n>Existing backdoor removal approaches rely on costly fine-tuning to override the harmful behavior.<n>This raises the question of whether backdoors can be removed without compromising the general capabilities of the models.
  arXiv  Detail & Related papers  (2025-10-16T16:18:07Z)
• SafeSplit: A Novel Defense Against Client-Side Backdoor Attacks in Split Learning (Full Version) [53.16528046390881]
  Split Learning (SL) is a distributed deep learning approach enabling multiple clients and a server to collaboratively train and infer on a shared deep neural network (DNN) This paper presents SafeSplit, the first defense against client-side backdoor attacks in Split Learning (SL) It uses a two-fold analysis to identify client-induced changes and detect poisoned models.
  arXiv  Detail & Related papers  (2025-01-11T22:20:20Z)
• Mithridates: Auditing and Boosting Backdoor Resistance of Machine Learning Pipelines [16.712814763678495]
  Machine learning models trained on data from potentially untrusted sources are vulnerable to poisoning. We propose a universal, attack-agnostic resistance metric based on the minimum number of training inputs that must be compromised. We show that Mithridates increases resistance to multiple types of backdoor attacks by 3-5x with only a slight impact on accuracy.
  arXiv  Detail & Related papers  (2023-02-09T23:34:17Z)
• Backdoor Cleansing with Unlabeled Data [70.29989887008209]
  externally trained Deep Neural Networks (DNNs) can potentially be backdoor attacked. We propose a novel defense method that does not require training labels. Our method, trained without labels, is on-par with state-of-the-art defense methods trained using labels.
  arXiv  Detail & Related papers  (2022-11-22T06:29:30Z)
• CrowdGuard: Federated Backdoor Detection in Federated Learning [39.58317527488534]
  This paper presents a novel defense mechanism, CrowdGuard, that effectively mitigates backdoor attacks in Federated Learning. CrowdGuard employs a server-located stacked clustering scheme to enhance its resilience to rogue client feedback. The evaluation results demonstrate that CrowdGuard achieves a 100% True-Positive-Rate and True-Negative-Rate across various scenarios.
  arXiv  Detail & Related papers  (2022-10-14T11:27:49Z)
• Planting Undetectable Backdoors in Machine Learning Models [14.592078676445201]
  We show how a malicious learner can plant an undetectable backdoor into a classifier. Without the appropriate "backdoor key", the mechanism is hidden and cannot be detected by any computationally-bounded observer. We show two frameworks for planting undetectable backdoors, with incomparable guarantees.
  arXiv  Detail & Related papers  (2022-04-14T13:55:21Z)
• Check Your Other Door! Establishing Backdoor Attacks in the Frequency Domain [80.24811082454367]
  We show the advantages of utilizing the frequency domain for establishing undetectable and powerful backdoor attacks. We also show two possible defences that succeed against frequency-based backdoor attacks and possible ways for the attacker to bypass them.
  arXiv  Detail & Related papers  (2021-09-12T12:44:52Z)
• CRFL: Certifiably Robust Federated Learning against Backdoor Attacks [59.61565692464579]
  This paper provides the first general framework, Certifiably Robust Federated Learning (CRFL), to train certifiably robust FL models against backdoors. Our method exploits clipping and smoothing on model parameters to control the global model smoothness, which yields a sample-wise robustness certification on backdoors with limited magnitude.
  arXiv  Detail & Related papers  (2021-06-15T16:50:54Z)
• Black-box Detection of Backdoor Attacks with Limited Information and Data [56.0735480850555]
  We propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models.
  arXiv  Detail & Related papers  (2021-03-24T12:06:40Z)
• WAFFLE: Watermarking in Federated Learning [14.598085887204805]
  Federated learning is a distributed learning technique where machine learning models are trained on client devices. We present WAFFLE, the first approach to watermark models trained using federated learning. We show that WAFFLE efficiently embeds a resilient watermark into models incurring only negligible degradation in test accuracy.
  arXiv  Detail & Related papers  (2020-08-17T13:27:45Z)
• Scalable Backdoor Detection in Neural Networks [61.39635364047679]
  Deep learning models are vulnerable to Trojan attacks, where an attacker can install a backdoor during training time to make the resultant model misidentify samples contaminated with a small trigger patch. We propose a novel trigger reverse-engineering based approach whose computational complexity does not scale with the number of labels, and is based on a measure that is both interpretable and universal across different network and patch types. In experiments, we observe that our method achieves a perfect score in separating Trojaned models from pure models, which is an improvement over the current state-of-the art method.
  arXiv  Detail & Related papers  (2020-06-10T04:12:53Z)
• Blind Backdoors in Deep Learning Models [22.844973592524966]
  We investigate a new method for injecting backdoors into machine learning models, based on compromising the loss-value computation in the model-training code. We use it to demonstrate new classes of backdoors strictly more powerful than those in the prior literature. Our attack is blind: the attacker cannot modify the training data, nor observe the execution of his code, nor access the resulting model.
  arXiv  Detail & Related papers  (2020-05-08T02:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.