Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization

• URL: http://arxiv.org/abs/2509.08233v1
• Date: Wed, 10 Sep 2025 02:19:56 GMT
• Title: Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization
• Authors: Kai Yi,
• Abstract summary: dissertation explores strategies to improve communication efficiency, focusing on model compression, local training, and personalization.<n>We establish a unified framework for biased and unbiased compression operators with convergence guarantees.<n>We propose adaptive local training strategies that incorporate personalization to accelerate convergence and mitigate client drift.
• Score: 8.579148218325168
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Distributed and federated learning are essential paradigms for training models across decentralized data sources while preserving privacy, yet communication overhead remains a major bottleneck. This dissertation explores strategies to improve communication efficiency, focusing on model compression, local training, and personalization. We establish a unified framework for biased and unbiased compression operators with convergence guarantees, then propose adaptive local training strategies that incorporate personalization to accelerate convergence and mitigate client drift. In particular, Scafflix balances global and personalized objectives, achieving superior performance under both IID and non-IID settings. We further introduce privacy-preserving pruning frameworks that optimize sparsity while minimizing communication costs, with Cohort-Squeeze leveraging hierarchical aggregation to reduce cross-device overhead. Finally, SymWanda, a symmetric post-training pruning method, enhances robustness under high sparsity and maintains accuracy without retraining. Extensive experiments on benchmarks and large-scale language models demonstrate favorable trade-offs among accuracy, convergence, and communication, offering theoretical and practical insights for scalable, efficient distributed learning.

Related papers

• FedCCL: Federated Clustered Continual Learning Framework for Privacy-focused Energy Forecasting [0.0]
  FedCCL is a framework specifically designed for environments with static organizational characteristics but dynamic client availability.<n>Our approach implements an asynchronous Federated Learning protocol with a three-tier model topology.<n>We show that FedCCL offers an effective framework for privacy-preserving distributed learning, maintaining high accuracy and adaptability even with dynamic participant populations.
  arXiv  Detail & Related papers  (2025-04-28T21:51:27Z)
• FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models [52.13056951033747]
  Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server.<n>A critical bottleneck in FL is the communication cost.<n>Our work is inspired by the innovative Scaffnew algorithm, which has advanced the reduction of communication complexity in FL.<n>We introduce FedComLoc, integrating practical and effective compression into Scaffnew to further enhance communication efficiency.
  arXiv  Detail & Related papers  (2024-03-14T22:29:59Z)
• Compressed Regression over Adaptive Networks [58.79251288443156]
  We derive the performance achievable by a network of distributed agents that solve, adaptively and in the presence of communication constraints, a regression problem. We devise an optimized allocation strategy where the parameters necessary for the optimization can be learned online by the agents.
  arXiv  Detail & Related papers  (2023-04-07T13:41:08Z)
• FedAgg: Adaptive Federated Learning with Aggregated Gradients [1.5653612447564105]
  We propose an adaptive FEDerated learning algorithm called FedAgg to alleviate the divergence between the local and average model parameters and obtain a fast model convergence rate. We show that our framework is superior to existing state-of-the-art FL strategies for enhancing model performance and accelerating convergence rate under IID and Non-IID datasets.
  arXiv  Detail & Related papers  (2023-03-28T08:07:28Z)
• Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
  Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
  arXiv  Detail & Related papers  (2023-02-24T08:41:19Z)
• FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning [87.08902493524556]
  Federated learning(FL) has recently attracted increasing attention from academia and industry. We propose FedDM to build the global training objective from multiple local surrogate functions. In detail, we construct synthetic sets of data on each client to locally match the loss landscape from original data.
  arXiv  Detail & Related papers  (2022-07-20T04:55:18Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimization [21.81192774458227]
  One of the major bottlenecks is the large communication cost between the central server and the local workers. Our proposed distributed learning framework features an effective gradient gradient compression strategy.
  arXiv  Detail & Related papers  (2021-11-01T04:54:55Z)
• Communication-Efficient Federated Learning with Dual-Side Low-Rank Compression [8.353152693578151]
  Federated learning (FL) is a promising and powerful approach for training deep learning models without sharing the raw data of clients. We propose a new training method, referred to as federated learning with dual-side low-rank compression (FedDLR) We show that FedDLR outperforms the state-of-the-art solutions in terms of both the communication and efficiency.
  arXiv  Detail & Related papers  (2021-04-26T09:13:31Z)
• CosSGD: Nonlinear Quantization for Communication-efficient Federated Learning [62.65937719264881]
  Federated learning facilitates learning across clients without transferring local data on these clients to a central server. We propose a nonlinear quantization for compressed gradient descent, which can be easily utilized in federated learning. Our system significantly reduces the communication cost by up to three orders of magnitude, while maintaining convergence and accuracy of the training process.
  arXiv  Detail & Related papers  (2020-12-15T12:20:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.