Privacy-Preserving Aggregation for Decentralized Learning with Byzantine-Robustness

• URL: http://arxiv.org/abs/2404.17970v1
• Date: Sat, 27 Apr 2024 18:17:36 GMT
• Title: Privacy-Preserving Aggregation for Decentralized Learning with Byzantine-Robustness
• Authors: Ali Reza Ghavamipour, Benjamin Zi Hao Zhao, Oguzhan Ersoy, Fatih Turkmen,
• Abstract summary: Byzantine clients intentionally disrupt the learning process by broadcasting arbitrary model updates to other clients. In this paper, we introduce SecureDL, a novel DL protocol designed to enhance the security and privacy of DL against Byzantine threats. Our experiments show that SecureDL is effective even in the case of attacks by the malicious majority.
• Score: 5.735144760031169
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Decentralized machine learning (DL) has been receiving an increasing interest recently due to the elimination of a single point of failure, present in Federated learning setting. Yet, it is threatened by the looming threat of Byzantine clients who intentionally disrupt the learning process by broadcasting arbitrary model updates to other clients, seeking to degrade the performance of the global model. In response, robust aggregation schemes have emerged as promising solutions to defend against such Byzantine clients, thereby enhancing the robustness of Decentralized Learning. Defenses against Byzantine adversaries, however, typically require access to the updates of other clients, a counterproductive privacy trade-off that in turn increases the risk of inference attacks on those same model updates. In this paper, we introduce SecureDL, a novel DL protocol designed to enhance the security and privacy of DL against Byzantine threats. SecureDL~facilitates a collaborative defense, while protecting the privacy of clients' model updates through secure multiparty computation. The protocol employs efficient computation of cosine similarity and normalization of updates to robustly detect and exclude model updates detrimental to model convergence. By using MNIST, Fashion-MNIST, SVHN and CIFAR-10 datasets, we evaluated SecureDL against various Byzantine attacks and compared its effectiveness with four existing defense mechanisms. Our experiments show that SecureDL is effective even in the case of attacks by the malicious majority (e.g., 80% Byzantine clients) while preserving high training accuracy.

Related papers

• FedCAP: Robust Federated Learning via Customized Aggregation and Personalization [13.17735010891312]
  Federated learning (FL) has been applied to various privacy-preserving scenarios. We propose FedCAP, a robust FL framework against both data heterogeneity and Byzantine attacks. We show that FedCAP performs well in several non-IID settings and shows strong robustness under a series of poisoning attacks.
  arXiv  Detail & Related papers  (2024-10-16T23:01:22Z)
• PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learning [49.916365792036636]
  Federated learning (FL) has recently gained significant momentum due to its potential to leverage large-scale distributed user data. The transmitted model updates can potentially leak sensitive user information, and the lack of central control of the local training process leaves the global model susceptible to malicious manipulations on model updates. We develop a general framework PriRoAgg, utilizing Lagrange coded computing and distributed zero-knowledge proof, to execute a wide range of robust aggregation algorithms while satisfying aggregated privacy.
  arXiv  Detail & Related papers  (2024-07-12T03:18:08Z)
• Here's a Free Lunch: Sanitizing Backdoored Models with Model Merge [17.3048898399324]
  democratization of pre-trained language models through open-source initiatives has rapidly advanced innovation and expanded access to cutting-edge technologies. backdoor attacks, where hidden malicious behaviors are triggered by specific inputs, compromising natural language processing (NLP) system integrity and reliability. This paper suggests that merging a backdoored model with other homogeneous models can significantly remediate backdoor vulnerabilities.
  arXiv  Detail & Related papers  (2024-02-29T16:37:08Z)
• Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacy [6.343100139647636]
  Federated learning (FL) is designed to preserve data privacy during model training. FL is vulnerable to privacy attacks and Byzantine attacks. We propose a new FL scheme that guarantees rigorous privacy and simultaneously enhances system robustness against Byzantine attacks.
  arXiv  Detail & Related papers  (2023-09-07T01:39:02Z)
• FedDefender: Client-Side Attack-Tolerant Federated Learning [60.576073964874]
  Federated learning enables learning from decentralized data sources without compromising privacy. It is vulnerable to model poisoning attacks, where malicious clients interfere with the training process. We propose a new defense mechanism that focuses on the client-side, called FedDefender, to help benign clients train robust local models.
  arXiv  Detail & Related papers  (2023-07-18T08:00:41Z)
• Robust Quantity-Aware Aggregation for Federated Learning [72.59915691824624]
  Malicious clients can poison model updates and claim large quantities to amplify the impact of their model updates in the model aggregation. Existing defense methods for FL, while all handling malicious model updates, either treat all quantities benign or simply ignore/truncate the quantities of all clients. We propose a robust quantity-aware aggregation algorithm for federated learning, called FedRA, to perform the aggregation with awareness of local data quantities.
  arXiv  Detail & Related papers  (2022-05-22T15:13:23Z)
• PRECAD: Privacy-Preserving and Robust Federated Learning via Crypto-Aided Differential Privacy [14.678119872268198]
  Federated Learning (FL) allows multiple participating clients to train machine learning models collaboratively by keeping their datasets local and only exchanging model updates. Existing FL protocol designs have been shown to be vulnerable to attacks that aim to compromise data privacy and/or model robustness. We develop a framework called PRECAD, which simultaneously achieves differential privacy (DP) and enhances robustness against model poisoning attacks with the help of cryptography.
  arXiv  Detail & Related papers  (2021-10-22T04:08:42Z)
• Secure Byzantine-Robust Distributed Learning via Clustering [16.85310886805588]
  Federated learning systems that jointly preserve Byzantine robustness and privacy have remained an open problem. We propose SHARE, a distributed learning framework designed to cryptographically preserve client update privacy and robustness to Byzantine adversaries simultaneously.
  arXiv  Detail & Related papers  (2021-10-06T17:40:26Z)
• RoFL: Attestable Robustness for Secure Federated Learning [59.63865074749391]
  Federated Learning allows a large number of clients to train a joint model without the need to share their private data. To ensure the confidentiality of the client updates, Federated Learning systems employ secure aggregation. We present RoFL, a secure Federated Learning system that improves robustness against malicious clients.
  arXiv  Detail & Related papers  (2021-07-07T15:42:49Z)
• Towards Bidirectional Protection in Federated Learning [70.36925233356335]
  F2ED-LEARNING offers bidirectional defense against malicious centralized server and Byzantine malicious clients. F2ED-LEARNING securely aggregates each shard's update and launches FilterL2 on updates from different shards. evaluation shows that F2ED-LEARNING consistently achieves optimal or close-to-optimal performance.
  arXiv  Detail & Related papers  (2020-10-02T19:37:02Z)
• Byzantine-resilient Decentralized Stochastic Gradient Descent [85.15773446094576]
  We present an in-depth study towards the Byzantine resilience of decentralized learning systems. We propose UBAR, a novel algorithm to enhance decentralized learning with Byzantine Fault Tolerance.
  arXiv  Detail & Related papers  (2020-02-20T05:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.