CYCle: Choosing Your Collaborators Wisely to Enhance Collaborative Fairness in Decentralized Learning

• URL: http://arxiv.org/abs/2501.12344v2
• Date: Wed, 01 Oct 2025 12:42:37 GMT
• Title: CYCle: Choosing Your Collaborators Wisely to Enhance Collaborative Fairness in Decentralized Learning
• Authors: Nurbek Tastan, Samuel Horvath, Karthik Nandakumar,
• Abstract summary: Collaborative learning (CL) enables multiple participants to jointly train machine learning (ML) models on decentralized data sources without raw data sharing.<n>While the primary goal of CL is to maximize expected accuracy gain for each participant, it is also important to ensure that the gains are fairly distributed.<n>Most existing CL methods require central coordination and focus only on gain, overlooking fairness.
• Score: 28.70691568233268
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Collaborative learning (CL) enables multiple participants to jointly train machine learning (ML) models on decentralized data sources without raw data sharing. While the primary goal of CL is to maximize the expected accuracy gain for each participant, it is also important to ensure that the gains are fairly distributed: no client should be negatively impacted, and gains should reflect contributions. Most existing CL methods require central coordination and focus only on gain maximization, overlooking fairness. In this work, we first show that the existing measure of collaborative fairness based on the correlation between accuracy values without and with collaboration has drawbacks because it does not account for negative collaboration gain. We argue that maximizing mean collaboration gain (MCG) while simultaneously minimizing the collaboration gain spread (CGS) is a fairer alternative. Next, we propose the CYCle protocol that enables individual participants in a private decentralized learning (PDL) framework to achieve this objective through a novel reputation scoring method based on gradient alignment between the local cross-entropy and distillation losses. We further extend the CYCle protocol to operate on top of gossip-based decentralized algorithms such as Gossip-SGD. We also theoretically show that CYCle performs better than standard FedAvg in a two-client mean estimation setting under high heterogeneity. Empirical experiments demonstrate the effectiveness of the CYCle protocol to ensure positive and fair collaboration gain for all participants, even in cases where the data distributions of participants are highly skewed.

Related papers

• Heterogeneous Agent Collaborative Reinforcement Learning [52.99813668995983]
  Heterogeneous Agent Collaborative Reinforcement Learning (HACRL)<n>Building on this paradigm, we propose HACPO, a collaborative RL algorithm that enables principled rollout sharing to maximize sample utilization and cross-agent knowledge transfer.<n>Experiments across diverse heterogeneous model combinations and reasoning benchmarks show that HACPO consistently improves all participating agents, outperforming GSPO by an average of 3.3% while using only half the rollout cost.
  arXiv  Detail & Related papers  (2026-03-03T05:09:49Z)
• Hybrid Federated and Split Learning for Privacy Preserving Clinical Prediction and Treatment Optimization [0.0]
  Collaborative clinical decision support is often constrained by governance and privacy rules that prevent pooling patient-level records across institutions.<n>We present a hybrid privacy-preserving framework that combines Federated Learning (FL) and Split Learning (SL) to support decision-oriented healthcare modeling without raw-data sharing.
  arXiv  Detail & Related papers  (2026-02-17T01:57:27Z)
• Toward a Sustainable Federated Learning Ecosystem: A Practical Least Core Mechanism for Payoff Allocation [71.86087908416255]
  We introduce a payoff allocation framework based on the least core (LC) concept.<n>Unlike traditional methods, the LC prioritizes the cohesion of the federation by minimizing the maximum dissatisfaction.<n>Case studies in federated intrusion detection demonstrate that our mechanism correctly identifies pivotal contributors and strategic alliances.
  arXiv  Detail & Related papers  (2026-02-03T11:10:50Z)
• CoRe-Fed: Bridging Collaborative and Representation Fairness via Federated Embedding Distillation [12.707158627881968]
  Federated Learning (FL) has emerged as a key approach to enable collaborative intelligence through decentralized model training.<n>We propose CoRe-Fed, a unified optimization framework that bridges collaborative and representation fairness.<n>We show that CoRe-Fed improves both fairness and model performance over the state-of-the-art baseline algorithms.
  arXiv  Detail & Related papers  (2026-01-31T10:41:00Z)
• Learning to Collaborate: An Orchestrated-Decentralized Framework for Peer-to-Peer LLM Federation [0.9231476323484774]
  This paper introduces KNEXA-FL, a novel framework for orchestrated decentralization.<n>It employs a non-aggregating Central Profiler/Matchmaker that formulates P2P collaboration as a contextual bandit problem.<n>Our experiments show that KNEXA-FL yields substantial gains, improving Pass@1 by approx. 50% relative to random P2P collaboration.
  arXiv  Detail & Related papers  (2026-01-23T19:18:43Z)
• Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learning [50.56947843548702]
  We propose a decentralized dynamic cooperation framework for Federated continual learning.<n>Clients establish dynamic cooperative learning coalitions to balance the acquisition of new knowledge and the retention of prior learning.<n>We also propose a merge-blocking algorithm and a dynamic cooperative evolution algorithm to achieve cooperative and dynamic equilibrium.
  arXiv  Detail & Related papers  (2025-09-28T06:53:23Z)
• FedSAC: Dynamic Submodel Allocation for Collaborative Fairness in Federated Learning [46.30755524556465]
  We present FedSAC, a novel Federated learning framework with dynamic Submodel Allocation for Collaborative fairness. We develop a submodel allocation module with a theoretical guarantee of fairness. Experiments conducted on three public benchmarks demonstrate that FedSAC outperforms all baseline methods in both fairness and model accuracy.
  arXiv  Detail & Related papers  (2024-05-28T15:43:29Z)
• Emulating Full Participation: An Effective and Fair Client Selection Strategy for Federated Learning [50.060154488277036]
  In federated learning, client selection is a critical problem that significantly impacts both model performance and fairness. We propose two guiding principles that tackle the inherent conflict between the two metrics while reinforcing each other. Our approach adaptively enhances this diversity by selecting clients based on their data distributions, thereby improving both model performance and fairness.
  arXiv  Detail & Related papers  (2024-05-22T12:27:24Z)
• Balancing Similarity and Complementarity for Federated Learning [91.65503655796603]
  Federated Learning (FL) is increasingly important in mobile and IoT systems. One key challenge in FL is managing statistical heterogeneity, such as non-i.i.d. data. We introduce a novel framework, textttFedSaC, which balances similarity and complementarity in FL cooperation.
  arXiv  Detail & Related papers  (2024-05-16T08:16:19Z)
• Federated Learning Can Find Friends That Are Advantageous [14.993730469216546]
  In Federated Learning (FL), the distributed nature and heterogeneity of client data present both opportunities and challenges. We introduce a novel algorithm that assigns adaptive aggregation weights to clients participating in FL training, identifying those with data distributions most conducive to a specific learning objective.
  arXiv  Detail & Related papers  (2024-02-07T17:46:37Z)
• How to Collaborate: Towards Maximizing the Generalization Performance in Cross-Silo Federated Learning [11.442808208742758]
  Federated clustering (FL) has vivid attention as a privacy-preserving distributed learning framework.<n>In this work, we focus on cross-silo FL, where clients become the model owners after FL data.<n>We formulate that the performance of a client can be improved only by collaborating with other clients that have more training data.
  arXiv  Detail & Related papers  (2024-01-24T05:41:34Z)
• Efficient Core-selecting Incentive Mechanism for Data Sharing in Federated Learning [0.12289361708127873]
  Federated learning is a distributed machine learning system that uses participants' data to train an improved global model. How to establish an incentive mechanism that both incentivizes inputting data truthfully and promotes stable cooperation has become an important issue to consider. We propose an efficient core-selecting mechanism based on sampling approximation that only aggregates models on sampled coalitions to approximate the exact result.
  arXiv  Detail & Related papers  (2023-09-21T01:47:39Z)
• Serverless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining [119.89373423433804]
  Area Under Precision-Recall (AUPRC) was introduced as an effective metric. Serverless multi-party collaborative training can cut down the communications cost by avoiding the server node bottleneck. We propose a new ServerLess biAsed sTochastic gradiEnt (SLATE) algorithm to directly optimize the AUPRC.
  arXiv  Detail & Related papers  (2023-08-06T06:51:32Z)
• Collaborative Learning via Prediction Consensus [38.89001892487472]
  We consider a collaborative learning setting where the goal of each agent is to improve their own model by leveraging the expertise of collaborators. We propose a distillation-based method leveraging shared unlabeled auxiliary data, which is pseudo-labeled by the collective. We demonstrate empirically that our collaboration scheme is able to significantly boost the performance of individual models.
  arXiv  Detail & Related papers  (2023-05-29T14:12:03Z)
• Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction [76.26710990597498]
  We show that the class-imbalance of the grouped data from randomly selected clients can lead to significant performance degradation. Based on our key observation, we design an efficient client sampling mechanism, i.e., Federated Class-balanced Sampling (Fed-CBS) In particular, we propose a measure of class-imbalance and then employ homomorphic encryption to derive this measure in a privacy-preserving way.
  arXiv  Detail & Related papers  (2022-09-30T05:42:56Z)
• Semi-supervised Contrastive Learning with Similarity Co-calibration [72.38187308270135]
  We propose a novel training strategy, termed as Semi-supervised Contrastive Learning (SsCL) SsCL combines the well-known contrastive loss in self-supervised learning with the cross entropy loss in semi-supervised learning. We show that SsCL produces more discriminative representation and is beneficial to few shot learning.
  arXiv  Detail & Related papers  (2021-05-16T09:13:56Z)
• Collaborative Fairness in Federated Learning [24.7378023761443]
  We propose a novel Collaborative Fair Federated Learning (CFFL) framework for deep learning. CFFL enforces participants to converge to different models, thus achieving fairness without compromising predictive performance. Experiments on benchmark datasets demonstrate that CFFL achieves high fairness and delivers comparable accuracy to the Distributed framework.
  arXiv  Detail & Related papers  (2020-08-27T14:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.