On Feasibility of Server-side Backdoor Attacks on Split Learning
- URL: http://arxiv.org/abs/2302.09578v2
- Date: Fri, 26 May 2023 07:27:20 GMT
- Title: On Feasibility of Server-side Backdoor Attacks on Split Learning
- Authors: Behrad Tajalli, Oguzhan Ersoy, Stjepan Picek
- Abstract summary: Split learning is a collaborative learning design that allows several participants (clients) to train a shared model while keeping their datasets private.
Recent studies demonstrate that collaborative learning models are vulnerable to security and privacy attacks such as model inference and backdoor attacks.
This paper performs a novel backdoor attack on split learning and studies its effectiveness.
- Score: 5.559334420715782
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Split learning is a collaborative learning design that allows several
participants (clients) to train a shared model while keeping their datasets
private. Recent studies demonstrate that collaborative learning models,
specifically federated learning, are vulnerable to security and privacy attacks
such as model inference and backdoor attacks. Backdoor attacks are a group of
poisoning attacks in which the attacker tries to control the model output by
manipulating the model's training process. While there have been studies
regarding inference attacks on split learning, it has not yet been tested for
backdoor attacks. This paper performs a novel backdoor attack on split learning
and studies its effectiveness. Despite traditional backdoor attacks done on the
client side, we inject the backdoor trigger from the server side. For this
purpose, we provide two attack methods: one using a surrogate client and
another using an autoencoder to poison the model via incoming smashed data and
its outgoing gradient toward the innocent participants. We did our experiments
using three model architectures and three publicly available datasets in the
image domain and ran a total of 761 experiments to evaluate our attack methods.
The results show that despite using strong patterns and injection methods,
split learning is highly robust and resistant to such poisoning attacks. While
we get the attack success rate of 100% as our best result for the MNIST
dataset, in most of the other cases, our attack shows little success when
increasing the cut layer.
Related papers
- Efficient Backdoor Defense in Multimodal Contrastive Learning: A Token-Level Unlearning Method for Mitigating Threats [52.94388672185062]
We propose an efficient defense mechanism against backdoor threats using a concept known as machine unlearning.
This entails strategically creating a small set of poisoned samples to aid the model's rapid unlearning of backdoor vulnerabilities.
In the backdoor unlearning process, we present a novel token-based portion unlearning training regime.
arXiv Detail & Related papers (2024-09-29T02:55:38Z) - DLP: towards active defense against backdoor attacks with decoupled learning process [2.686336957004475]
We propose a general training pipeline to defend against backdoor attacks.
We show that the model shows different learning behaviors in clean and poisoned subsets during training.
The effectiveness of our approach has been shown in numerous experiments across various backdoor attacks and datasets.
arXiv Detail & Related papers (2024-06-18T23:04:38Z) - Rethinking Backdoor Attacks [122.1008188058615]
In a backdoor attack, an adversary inserts maliciously constructed backdoor examples into a training set to make the resulting model vulnerable to manipulation.
Defending against such attacks typically involves viewing these inserted examples as outliers in the training set and using techniques from robust statistics to detect and remove them.
We show that without structural information about the training data distribution, backdoor attacks are indistinguishable from naturally-occurring features in the data.
arXiv Detail & Related papers (2023-07-19T17:44:54Z) - Backdoor Learning on Sequence to Sequence Models [94.23904400441957]
In this paper, we study whether sequence-to-sequence (seq2seq) models are vulnerable to backdoor attacks.
Specifically, we find by only injecting 0.2% samples of the dataset, we can cause the seq2seq model to generate the designated keyword and even the whole sentence.
Extensive experiments on machine translation and text summarization have been conducted to show our proposed methods could achieve over 90% attack success rate on multiple datasets and models.
arXiv Detail & Related papers (2023-05-03T20:31:13Z) - On the Effectiveness of Adversarial Training against Backdoor Attacks [111.8963365326168]
A backdoored model always predicts a target class in the presence of a predefined trigger pattern.
In general, adversarial training is believed to defend against backdoor attacks.
We propose a hybrid strategy which provides satisfactory robustness across different backdoor attacks.
arXiv Detail & Related papers (2022-02-22T02:24:46Z) - Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks [58.0225587881455]
In this paper, we find two simple tricks that can make existing textual backdoor attacks much more harmful.
The first trick is to add an extra training task to distinguish poisoned and clean data during the training of the victim model.
The second one is to use all the clean training data rather than remove the original clean data corresponding to the poisoned data.
arXiv Detail & Related papers (2021-10-15T17:58:46Z) - Widen The Backdoor To Let More Attackers In [24.540853975732922]
We investigate the scenario of a multi-agent backdoor attack, where multiple non-colluding attackers craft and insert triggered samples in a shared dataset.
We discover a clear backfiring phenomenon: increasing the number of attackers shrinks each attacker's attack success rate.
We then exploit this phenomenon to minimize the collective ASR of attackers and maximize defender's robustness accuracy.
arXiv Detail & Related papers (2021-10-09T13:53:57Z) - Check Your Other Door! Establishing Backdoor Attacks in the Frequency
Domain [80.24811082454367]
We show the advantages of utilizing the frequency domain for establishing undetectable and powerful backdoor attacks.
We also show two possible defences that succeed against frequency-based backdoor attacks and possible ways for the attacker to bypass them.
arXiv Detail & Related papers (2021-09-12T12:44:52Z) - Backdoor attacks and defenses in feature-partitioned collaborative
learning [11.162867684516995]
We show that even parties with no access to labels can successfully inject backdoor attacks.
This is the first systematical study to deal with backdoor attacks in the feature-partitioned collaborative learning framework.
arXiv Detail & Related papers (2020-07-07T16:45:20Z) - Backdoor Attacks on Federated Meta-Learning [0.225596179391365]
We analyze the effects of backdoor attacks on federated meta-learning.
We propose a defense mechanism inspired by matching networks, where the class of an input is predicted from the similarity of its features.
arXiv Detail & Related papers (2020-06-12T09:23:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.