A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraints

• URL: http://arxiv.org/abs/2608.26493v2
• Date: Sat, 29 Aug 2026 23:27:59 GMT
• Title: A Unified Framework for Fair and Personalized Decentralized Learning under Communication Constraints
• Abstract summary: Decentralized learning systems aim to collaboratively train models across multiple clients without relying on a central coordinator.<n>While decentralization improves scalability, privacy, and robustness, it also exacerbates three fundamental challenges: statistical heterogeneity across clients, synchronization in client-level performance, and stringent communication constraints.<n>Specifically, we propose a new algorithm DMFL-SQ, a decentralized multi-task learning algorithm that agnostic personalized model over a communication graph with an objective, while reducing sparsification through quantization.<n>We show that DMFL-SQ substantially reduces communication while maintaining predictive performance and improving fairness across clients.
• Score: 12.544579930854447
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Decentralized learning systems aim to collaboratively train models across multiple clients without relying on a central coordinator. While decentralization improves scalability, privacy, and robustness, it also exacerbates three fundamental challenges: statistical heterogeneity across clients, fairness in client-level performance, and stringent communication constraints. This raises a natural question: \emph{how fair can decentralized learning be under limited communication?} We address this question by presenting a unified framework for decentralized learning under communication constraints, bringing together graph-based personalization, agnostic fairness, and compressed event-triggered communication. Specifically, we propose a new algorithm DMFL-SQ, a decentralized multi-task learning algorithm that couples personalized model training over a communication graph with an agnostic mixture fairness objective, while reducing communication through sparsification, quantization, and event-triggered synchronization. We establish convergence guarantees for general non-convex objectives and show that DMFL-SQ achieves an $\mathcal{O}(T^{-1/2})$ rate in expected squared Moreau-envelope stationarity despite sparse, quantized, and event-triggered communication. We further derive PAC-Bayes generalization guarantees for the fairness-aware mixture objective. Experiments on CIFAR-10 and the real heterogeneous MUSMET EEG dataset demonstrate that DMFL-SQ substantially reduces communication while maintaining predictive performance and improving fairness across clients. Together, our theoretical and empirical results show that personalization, fairness, and communication efficiency can be jointly achieved in decentralized learning while preserving the dominant convergence rate.

Related papers

• Event-Triggered Gossip for Distributed Learning [61.70659996356528]
  We develop a new event-triggered gossip framework for distributed learning to reduce inter-node communication.<n>We analyze bf71.61% with only a marginal performance loss, compared with the conventional full-text-of-the-art distributed learning methods.
  arXiv  Detail & Related papers  (2026-02-22T10:13:43Z)
• Personalized Federated Learning with Bidirectional Communication Compression via One-Bit Random Sketching [19.959825317183924]
  Federated Learning (FL) enables collaborative training across decentralized data.<n>We propose pFed1BS, a novel personalized FL framework that achieves extreme communication compression through one-bit random sketching.<n>In our framework, clients transmit highly compressed one-bit sketches, and the server aggregates and broadcasts a global one-bit consensus.
  arXiv  Detail & Related papers  (2025-11-17T08:55:22Z)
• Zero-Shot Decentralized Federated Learning [10.60647737661699]
  We propose Zero-shot Decentralized Federated Learning (ZeroDFL), a fully decentralized framework that enables zero-shot adaptation across distributed clients without a central coordinator.<n>We validate ZeroDFL on nine diverse image classification datasets, demonstrating that it consistently outperforms--or remains on par with--state-of-the-art federated prompt learning methods.<n>These results highlight that our approach not only enhances generalization in federated zero-shot learning but also improves scalability, efficiency, and privacy preservation--paving the way for decentralized adaptation of large vision-language models in real-world applications.
  arXiv  Detail & Related papers  (2025-09-30T16:13:21Z)
• Strategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization [8.579148218325168]
  dissertation explores strategies to improve communication efficiency, focusing on model compression, local training, and personalization.<n>We establish a unified framework for biased and unbiased compression operators with convergence guarantees.<n>We propose adaptive local training strategies that incorporate personalization to accelerate convergence and mitigate client drift.
  arXiv  Detail & Related papers  (2025-09-10T02:19:56Z)
• MAB-Based Channel Scheduling for Asynchronous Federated Learning in Non-Stationary Environments [12.404264058659429]
  Federated learning enables distributed model training across clients without raw data exchange.<n>In wireless implementations, frequent parameter updates cause high communication overhead.<n>We propose an asynchronous federated learning scheduling framework to reduce client staleness while enhancing communication efficiency and fairness.
  arXiv  Detail & Related papers  (2025-03-03T09:05:04Z)
• Boosting the Performance of Decentralized Federated Learning via Catalyst Acceleration [66.43954501171292]
  We introduce Catalyst Acceleration and propose an acceleration Decentralized Federated Learning algorithm called DFedCata. DFedCata consists of two main components: the Moreau envelope function, which addresses parameter inconsistencies, and Nesterov's extrapolation step, which accelerates the aggregation phase. Empirically, we demonstrate the advantages of the proposed algorithm in both convergence speed and generalization performance on CIFAR10/100 with various non-iid data distributions.
  arXiv  Detail & Related papers  (2024-10-09T06:17:16Z)
• FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models [52.13056951033747]
  Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server.<n>A critical bottleneck in FL is the communication cost.<n>Our work is inspired by the innovative Scaffnew algorithm, which has advanced the reduction of communication complexity in FL.<n>We introduce FedComLoc, integrating practical and effective compression into Scaffnew to further enhance communication efficiency.
  arXiv  Detail & Related papers  (2024-03-14T22:29:59Z)
• DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training [84.81043932706375]
  We propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL. Dis-PFL employs personalized sparse masks to customize sparse local models on the edge. We demonstrate that our method can easily adapt to heterogeneous local clients with varying computation complexities.
  arXiv  Detail & Related papers  (2022-06-01T02:20:57Z)
• Finite-Time Consensus Learning for Decentralized Optimization with Nonlinear Gossiping [77.53019031244908]
  We present a novel decentralized learning framework based on nonlinear gossiping (NGO), that enjoys an appealing finite-time consensus property to achieve better synchronization. Our analysis on how communication delay and randomized chats affect learning further enables the derivation of practical variants.
  arXiv  Detail & Related papers  (2021-11-04T15:36:25Z)
• CosSGD: Nonlinear Quantization for Communication-efficient Federated Learning [62.65937719264881]
  Federated learning facilitates learning across clients without transferring local data on these clients to a central server. We propose a nonlinear quantization for compressed gradient descent, which can be easily utilized in federated learning. Our system significantly reduces the communication cost by up to three orders of magnitude, while maintaining convergence and accuracy of the training process.
  arXiv  Detail & Related papers  (2020-12-15T12:20:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.