Byzantine-Robust and Communication-Efficient Distributed Training: Compressive and Cyclic Gradient Coding

• URL: http://arxiv.org/abs/2603.28780v1
• Date: Tue, 17 Mar 2026 10:22:04 GMT
• Title: Byzantine-Robust and Communication-Efficient Distributed Training: Compressive and Cyclic Gradient Coding
• Abstract summary: We study the problem of distributed training (DT) under Byzantine attacks with communication constraints.<n>We propose a novel DT method, cyclic gradient coding-based DT (LAD)<n>LAD demonstrates improved robustness against Byzantine attacks and significantly lower solution error.
• Score: 39.5537175388854
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we study the problem of distributed training (DT) under Byzantine attacks with communication constraints. While prior work has developed various robust aggregation rules at the server to enhance robustness to Byzantine attacks, the existing methods suffer from a critical limitation in that the solution error does not diminish when the local gradients sent by different devices vary considerably, as a result of data heterogeneity among the subsets held by different devices. To overcome this limitation, we propose a novel DT method, cyclic gradient coding-based DT (LAD). In LAD, the server allocates the entire training dataset to the devices before training begins. In each iteration, it assigns computational tasks redundantly to the devices using cyclic gradient coding. Each honest device then computes local gradients on a fixed number of data subsets and encodes the local gradients before transmitting to the server. The server aggregates the coded vectors from the honest devices and the potentially incorrect messages from Byzantine devices using a robust aggregation rule. Leveraging the redundancy of computation across devices, the convergence performance of LAD is analytically characterized, demonstrating improved robustness against Byzantine attacks and significantly lower solution error. Furthermore, we extend LAD to a communication-efficient variant, compressive and cyclic gradient coding-based DT (Com-LAD), which further reduces communication overhead under constrained settings. Numerical results validate the effectiveness of the proposed methods in enhancing both Byzantine resilience and communication efficiency.

Related papers

• Provable Sparse Inversion and Token Relabel Enhanced One-shot Federated Learning with ViTs [77.42033827176806]
  One-Shot Federated Learning, where a central server learns a global model in a single communication round, has emerged as a promising paradigm.<n>We propose a novel Federated Model Inversion and Token Relabel framework, which trains the global model by fully exploiting all patches of synthetic images.
  arXiv  Detail & Related papers  (2026-05-11T15:49:08Z)
• Prioritizing Gradient Sign Over Modulus: An Importance-Aware Framework for Wireless Federated Learning [72.18619424901249]
  Sign-Prioritized FL (SP-FL) is a framework that prioritizes the transmission of important gradient information through uneven resource allocation.<n>We propose an alternating optimization algorithm to solve this problem using the Newton-Raphson method and successive convex approximation (SCA)<n> Simulation results confirm the superiority of SP-FL, especially in resource-constrained scenarios.
  arXiv  Detail & Related papers  (2026-03-11T13:41:02Z)
• Accelerate Speculative Decoding with Sparse Computation in Verification [49.74839681322316]
  Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel.<n>Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding.<n>We propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost.
  arXiv  Detail & Related papers  (2025-12-26T07:53:41Z)
• SemanticNN: Compressive and Error-Resilient Semantic Offloading for Extremely Weak Devices [9.795432423267503]
  We propose SemanticNN, a semanticcoder that tolerates bit-level errors in pursuit of semantic-level correctness.<n>It incorporates a Bit Error Rate (BER)-aware decoder that adapts to dynamic channel conditions and a Soft Quantization (SQ)-based encoder to learn compact representations.<n>We conduct extensive experiments on STM32 using three models and six datasets across image classification and object detection tasks.
  arXiv  Detail & Related papers  (2025-11-14T07:47:25Z)
• Integrated Sensing, Communication, and Computation for Over-the-Air Federated Edge Learning [52.904670248426626]
  This paper studies an over-the-air federated edge learning (Air-FEEL) system with integrated sensing, communication, and computation.<n>We derive a low-complexity I SCC algorithm by alternately optimizing the batch size control and the network resource allocation.
  arXiv  Detail & Related papers  (2025-08-21T02:46:46Z)
• Coded Robust Aggregation for Distributed Learning under Byzantine Attacks [33.56146654796337]
  We propose a new DL method to cope with Byzantine attacks based on coded robust aggregation (CRA-DL)<n>CRA-DL is improved by the fact that the coded gradients sent by the honest devices are closer to each other.
  arXiv  Detail & Related papers  (2025-05-17T12:06:04Z)
• FedScalar: A Communication efficient Federated Learning [0.0]
  Federated learning (FL) has gained considerable popularity for distributed machine learning. emphFedScalar enables agents to communicate updates using a single scalar.
  arXiv  Detail & Related papers  (2024-10-03T07:06:49Z)
• Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AI [108.08079323459822]
  This paper studies a new multi-intelligent edge artificial-latency (AI) system, which jointly exploits the AI model split inference and integrated sensing and communication (ISAC) We measure the inference accuracy by adopting an approximate but tractable metric, namely discriminant gain.
  arXiv  Detail & Related papers  (2022-07-03T06:57:07Z)
• Communication-Efficient Gradient Coding for Straggler Mitigation in Distributed Learning [17.454251607446555]
  Distributed implementations of gradient-based methods, wherein a server distributes gradient computations across worker machines, need to overcome two limitations. Ye and Abbe [ICML 2018] proposed a coding-theoretic paradigm to characterize a fundamental trade-off between computation load per worker, communication overhead per worker, and straggler tolerance. We develop a communication-efficient gradient coding framework to overcome these drawbacks.
  arXiv  Detail & Related papers  (2020-05-14T17:57:13Z)
• A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems [82.2513703281725]
  Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices. We present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems.
  arXiv  Detail & Related papers  (2020-03-18T05:56:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.