Understanding Byzantine Robustness in Federated Learning with A Black-box Server

• URL: http://arxiv.org/abs/2408.06042v1
• Date: Mon, 12 Aug 2024 10:18:24 GMT
• Title: Understanding Byzantine Robustness in Federated Learning with A Black-box Server
• Authors: Fangyuan Zhao, Yuexiang Xie, Xuebin Ren, Bolin Ding, Shusen Yang, Yaliang Li,
• Abstract summary: Federated learning (FL) becomes vulnerable to Byzantine attacks where some of participators tend to damage the utility or discourage the convergence of the learned model. Previous works propose to apply robust rules to aggregate updates from participators against different types of Byzantine attacks. In practice, FL systems can involve a black-box server that makes the adopted aggregation rule inaccessible to participants, which can naturally defend or weaken some Byzantine attacks.
• Score: 47.98788470469994
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) becomes vulnerable to Byzantine attacks where some of participators tend to damage the utility or discourage the convergence of the learned model via sending their malicious model updates. Previous works propose to apply robust rules to aggregate updates from participators against different types of Byzantine attacks, while at the same time, attackers can further design advanced Byzantine attack algorithms targeting specific aggregation rule when it is known. In practice, FL systems can involve a black-box server that makes the adopted aggregation rule inaccessible to participants, which can naturally defend or weaken some Byzantine attacks. In this paper, we provide an in-depth understanding on the Byzantine robustness of the FL system with a black-box server. Our investigation demonstrates the improved Byzantine robustness of a black-box server employing a dynamic defense strategy. We provide both empirical evidence and theoretical analysis to reveal that the black-box server can mitigate the worst-case attack impact from a maximum level to an expectation level, which is attributed to the inherent inaccessibility and randomness offered by a black-box server.The source code is available at https://github.com/alibaba/FederatedScope/tree/Byzantine_attack_defense to promote further research in the community.

Related papers

• Byzantine-Robust Federated Learning: An Overview With Focus on Developing Sybil-based Attacks to Backdoor Augmented Secure Aggregation Protocols [0.0]
  Federated Learning (FL) paradigms enable large numbers of clients to collaboratively train Machine Learning models on private data. Traditional FL schemes are left vulnerable to Byzantine attacks that attempt to hurt model performance by injecting malicious backdoors. This paper provides a exhaustive and updated taxonomy of existing methods and frameworks, before zooming in and conducting an in-depth analysis of the strengths and weaknesses of the Robustness of Federated Learning protocol. We propose two novel Sybil-based attacks that take advantage of vulnerabilities in RoFL.
  arXiv  Detail & Related papers  (2024-10-30T04:20:22Z)
• BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning [85.2564206440109]
  This paper reveals the threats in this practical scenario that backdoor attacks can remain effective even after defenses. We introduce the emphtoolns attack, which is resistant to backdoor detection and model fine-tuning defenses.
  arXiv  Detail & Related papers  (2023-11-20T02:21:49Z)
• Certifiable Black-Box Attacks with Randomized Adversarial Examples: Breaking Defenses with Provable Confidence [34.35162562625252]
  Black-box adversarial attacks have demonstrated strong potential to compromise machine learning models. We study a new paradigm of black-box attacks with provable guarantees. This new black-box attack unveils significant vulnerabilities of machine learning models.
  arXiv  Detail & Related papers  (2023-04-10T01:12:09Z)
• Byzantines can also Learn from History: Fall of Centered Clipping in Federated Learning [6.974212769362569]
  In this work, we introduce a novel attack strategy that can circumvent the defences of CC framework. We also propose a new robust and fast defence mechanism that is effective against the proposed and other existing Byzantine attacks.
  arXiv  Detail & Related papers  (2022-08-21T14:39:30Z)
• Parallel Rectangle Flip Attack: A Query-based Black-box Attack against Object Detection [89.08832589750003]
  We propose a Parallel Rectangle Flip Attack (PRFA) via random search to avoid sub-optimal detection near the attacked region. Our method can effectively and efficiently attack various popular object detectors, including anchor-based and anchor-free, and generate transferable adversarial examples.
  arXiv  Detail & Related papers  (2022-01-22T06:00:17Z)
• Certified Federated Adversarial Training [3.474871319204387]
  We tackle the scenario of securing FL systems conducting adversarial training when a quorum of workers could be completely malicious. We model an attacker who poisons the model to insert a weakness into the adversarial training such that the model displays apparent adversarial robustness. We show that this defence can preserve adversarial robustness even against an adaptive attacker.
  arXiv  Detail & Related papers  (2021-12-20T13:40:20Z)
• Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS [70.60975663021952]
  We study blackbox adversarial attacks on network classifiers. We argue that attacker-defender fixed points are themselves general-sum games with complex phase transitions. We show that a continual learning approach is required to study attacker-defender dynamics.
  arXiv  Detail & Related papers  (2021-11-23T23:42:16Z)
• Adversarial training may be a double-edged sword [50.09831237090801]
  We show that some geometric consequences of adversarial training on the decision boundary of deep networks give an edge to certain types of black-box attacks. In particular, we define a metric called robustness gain to show that while adversarial training is an effective method to dramatically improve the robustness in white-box scenarios, it may not provide such a good robustness gain against the more realistic decision-based black-box attacks.
  arXiv  Detail & Related papers  (2021-07-24T19:09:16Z)
• Blacklight: Scalable Defense for Neural Networks against Query-Based Black-Box Attacks [34.04323550970413]
  We propose Blacklight, a new defense against query-based black-box adversarial attacks. Blacklight detects query-based black-box attacks by detecting highly similar queries. We evaluate Blacklight against eight state-of-the-art attacks, across a variety of models and image classification tasks.
  arXiv  Detail & Related papers  (2020-06-24T20:52:24Z)
• Spanning Attack: Reinforce Black-box Attacks with Unlabeled Data [96.92837098305898]
  Black-box attacks aim to craft adversarial perturbations by querying input-output pairs of machine learning models. Black-box attacks often suffer from the issue of query inefficiency due to the high dimensionality of the input space. We propose a novel technique called the spanning attack, which constrains adversarial perturbations in a low-dimensional subspace via spanning an auxiliary unlabeled dataset.
  arXiv  Detail & Related papers  (2020-05-11T05:57:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.