Coded Robust Aggregation for Distributed Learning under Byzantine Attacks

• URL: http://arxiv.org/abs/2506.01989v1
• Date: Sat, 17 May 2025 12:06:04 GMT
• Title: Coded Robust Aggregation for Distributed Learning under Byzantine Attacks
• Abstract summary: We propose a new DL method to cope with Byzantine attacks based on coded robust aggregation (CRA-DL)<n>CRA-DL is improved by the fact that the coded gradients sent by the honest devices are closer to each other.
• Score: 33.56146654796337
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we investigate the problem of distributed learning (DL) in the presence of Byzantine attacks. For this problem, various robust bounded aggregation (RBA) rules have been proposed at the central server to mitigate the impact of Byzantine attacks. However, current DL methods apply RBA rules for the local gradients from the honest devices and the disruptive information from Byzantine devices, and the learning performance degrades significantly when the local gradients of different devices vary considerably from each other. To overcome this limitation, we propose a new DL method to cope with Byzantine attacks based on coded robust aggregation (CRA-DL). Before training begins, the training data are allocated to the devices redundantly. During training, in each iteration, the honest devices transmit coded gradients to the server computed from the allocated training data, and the server then aggregates the information received from both honest and Byzantine devices using RBA rules. In this way, the global gradient can be approximately recovered at the server to update the global model. Compared with current DL methods applying RBA rules, the improvement of CRA-DL is attributed to the fact that the coded gradients sent by the honest devices are closer to each other. This closeness enhances the robustness of the aggregation against Byzantine attacks, since Byzantine messages tend to be significantly different from those of honest devices in this case. We theoretically analyze the convergence performance of CRA-DL. Finally, we present numerical results to verify the superiority of the proposed method over existing baselines, showing its enhanced learning performance under Byzantine attacks.

Related papers

• FL-OA: A Byzantine-Robust Federated Learning Framework with Outsourced Auditing for Intelligent Devices [23.049429240260064]
  Federated learning (FL) enables multiple intelligent devices to collaboratively train a high-accuracy model without sharing raw data.<n>Due to its distributed nature, FL is vulnerable to Byzantine attacks.<n>We propose FL-OA, a Byzantine-robust federated learning framework utilizing outsourced auditing.
  arXiv  Detail & Related papers  (2026-08-02T08:50:52Z)
• Byzantine-Robust and Communication-Efficient Distributed Training: Compressive and Cyclic Gradient Coding [39.5537175388854]
  We study the problem of distributed training (DT) under Byzantine attacks with communication constraints.<n>We propose a novel DT method, cyclic gradient coding-based DT (LAD)<n>LAD demonstrates improved robustness against Byzantine attacks and significantly lower solution error.
  arXiv  Detail & Related papers  (2026-03-17T10:22:04Z)
• Data-regularized Reinforcement Learning for Diffusion Models at Scale [99.01056178660538]
  We introduce Data-regularized Diffusion Reinforcement Learning ( DDRL), a novel framework that uses the forward KL divergence to anchor the policy to an off-policy data distribution.<n>With over a million GPU hours of experiments and ten thousand double-blind evaluations, we demonstrate that DDRL significantly improves rewards while alleviating the reward hacking seen in RLs.
  arXiv  Detail & Related papers  (2025-12-03T23:45:07Z)
• Improving LLM Safety Alignment with Dual-Objective Optimization [65.41451412400609]
  Existing training-time safety alignment techniques for large language models (LLMs) remain vulnerable to jailbreak attacks.<n>We propose an improved safety alignment that disentangles DPO objectives into two components: (1) robust refusal training, which encourages refusal even when partial unsafe generations are produced, and (2) targeted unlearning of harmful knowledge.
  arXiv  Detail & Related papers  (2025-03-05T18:01:05Z)
• Adversarial Training for Defense Against Label Poisoning Attacks [53.893792844055106]
  Label poisoning attacks pose significant risks to machine learning models.<n>We propose a novel adversarial training defense strategy based on support vector machines (SVMs) to counter these threats.<n>Our approach accommodates various model architectures and employs a projected gradient descent algorithm with kernel SVMs for adversarial training.
  arXiv  Detail & Related papers  (2025-02-24T13:03:19Z)
• Privacy-Preserving Aggregation for Decentralized Learning with Byzantine-Robustness [5.735144760031169]
  Byzantine clients intentionally disrupt the learning process by broadcasting arbitrary model updates to other clients. In this paper, we introduce SecureDL, a novel DL protocol designed to enhance the security and privacy of DL against Byzantine threats. Our experiments show that SecureDL is effective even in the case of attacks by the malicious majority.
  arXiv  Detail & Related papers  (2024-04-27T18:17:36Z)
• Dynamic Byzantine-Robust Learning: Adapting to Switching Byzantine Workers [10.632248569865236]
  Byzantine-robust learning has emerged as a prominent fault-tolerant distributed machine learning framework. We propose DynaBRO -- a new method capable of withstanding any sub-linear number of identity changes across rounds. Our method utilizes a multi-level Monte Carlo gradient technique applied at the server to robustly aggregated worker updates.
  arXiv  Detail & Related papers  (2024-02-05T12:26:01Z)
• Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting [58.91947205027892]
  Federated learning has exhibited vulnerabilities to Byzantine attacks. Byzantine attackers can send arbitrary gradients to a central server to destroy the convergence and performance of the global model. A wealth of robust AGgregation Rules (AGRs) have been proposed to defend against Byzantine attacks.
  arXiv  Detail & Related papers  (2023-02-13T03:31:50Z)
• Security-Preserving Federated Learning via Byzantine-Sensitive Triplet Distance [10.658882342481542]
  Federated learning (FL) is generally vulnerable to Byzantine attacks from adversarial edge devices. We propose an effective Byzantine-robust FL framework, namely dummy contrastive aggregation. We show improved performance as compared to the state-of-the-art Byzantine-resilient aggregation methods.
  arXiv  Detail & Related papers  (2022-10-29T07:20:02Z)
• Linear Scalarization for Byzantine-robust learning on non-IID data [3.098066034546503]
  We study the problem of Byzantine-robust learning when data among clients is heterogeneous. We propose the use of Linear Scalarization (LS) as an enhancing method to enable current defenses to circumvent Byzantine attacks in the non-IID setting.
  arXiv  Detail & Related papers  (2022-10-15T13:24:00Z)
• Learning from History for Byzantine Robust Optimization [52.68913869776858]
  Byzantine robustness has received significant attention recently given its importance for distributed learning. We show that most existing robust aggregation rules may not converge even in the absence of any Byzantine attackers.
  arXiv  Detail & Related papers  (2020-12-18T16:22:32Z)
• Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks [74.36161581953658]
  This paper deals with distributed finite-sum optimization for learning over networks in the presence of malicious Byzantine attacks. To cope with such attacks, most resilient approaches so far combine gradient descent (SGD) with different robust aggregation rules. The present work puts forth a Byzantine attack resilient distributed (Byrd-) SAGA approach for learning tasks involving finite-sum optimization over networks.
  arXiv  Detail & Related papers  (2019-12-29T19:46:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.