Comparative Evaluation of Clustered Federated Learning Methods

• URL: http://arxiv.org/abs/2410.14212v2
• Date: Thu, 14 Nov 2024 08:31:10 GMT
• Title: Comparative Evaluation of Clustered Federated Learning Methods
• Authors: Michael Ben Ali, Omar El-Rifai, Imen Megdiche, André Peninou, Olivier Teste,
• Abstract summary: Clustered Federated Learning (CFL) aims to partition clients into groups where the distribution are homogeneous. In this paper, we explore the performance of two state-of-theart CFL algorithms with respect to a proposed taxonomy of data heterogeneities in federated learning (FL) Our objective is to provide a clearer understanding of the relationship between CFL performances and data heterogeneous scenarios.
• Score: 0.5242869847419834
• License:
• Abstract: Over recent years, Federated Learning (FL) has proven to be one of the most promising methods of distributed learning which preserves data privacy. As the method evolved and was confronted to various real-world scenarios, new challenges have emerged. One such challenge is the presence of highly heterogeneous (often referred as non-IID) data distributions among participants of the FL protocol. A popular solution to this hurdle is Clustered Federated Learning (CFL), which aims to partition clients into groups where the distribution are homogeneous. In the literature, state-of-the-art CFL algorithms are often tested using a few cases of data heterogeneities, without systematically justifying the choices. Further, the taxonomy used for differentiating the different heterogeneity scenarios is not always straightforward. In this paper, we explore the performance of two state-of-theart CFL algorithms with respect to a proposed taxonomy of data heterogeneities in federated learning (FL). We work with three image classification datasets and analyze the resulting clusters against the heterogeneity classes using extrinsic clustering metrics. Our objective is to provide a clearer understanding of the relationship between CFL performances and data heterogeneity scenarios.

Related papers

• A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning [14.466679488063217]
  One-shot federated learning (FL) limits the communication between the server and clients to a single round. We propose a unified, data-free, one-shot FL framework (FedHydra) that can effectively address both model and data heterogeneity.
  arXiv  Detail & Related papers  (2024-10-28T15:20:52Z)
• Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalization [35.48757125452761]
  Federated learning is an efficient framework designed to facilitate collaborative model training across multiple distributed devices. A significant challenge of federated learning is data-level heterogeneity, i.e., skewed or long-tailed distribution of private data. We propose a novel Federated Prototype Rectification with Personalization which consists of two parts: Federated Personalization and Federated Prototype Rectification.
  arXiv  Detail & Related papers  (2024-08-15T06:26:46Z)
• FedSym: Unleashing the Power of Entropy for Benchmarking the Algorithms for Federated Learning [1.4656078321003647]
  Federated learning (FL) is a decentralized machine learning approach where independent learners process data privately. We study the currently popular data partitioning techniques and visualize their main disadvantages. We propose a method that leverages entropy and symmetry to construct 'the most challenging' and controllable data distributions.
  arXiv  Detail & Related papers  (2023-10-11T18:39:08Z)
• Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learning [60.058083574671834]
  This paper presents a novel FCCL+, federated correlation and similarity learning with non-target distillation. For heterogeneous issue, we leverage irrelevant unlabeled public data for communication. For catastrophic forgetting in local updating stage, FCCL+ introduces Federated Non Target Distillation.
  arXiv  Detail & Related papers  (2023-09-28T09:32:27Z)
• Tackling Diverse Minorities in Imbalanced Classification [80.78227787608714]
  Imbalanced datasets are commonly observed in various real-world applications, presenting significant challenges in training classifiers. We propose generating synthetic samples iteratively by mixing data samples from both minority and majority classes. We demonstrate the effectiveness of our proposed framework through extensive experiments conducted on seven publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-28T18:48:34Z)
• Stochastic Clustered Federated Learning [21.811496586350653]
  This paper proposes StoCFL, a novel clustered federated learning approach for generic Non-IID issues. In detail, StoCFL implements a flexible CFL framework that supports an arbitrary proportion of client participation and newly joined clients. The results show that StoCFL could obtain promising cluster results even when the number of clusters is unknown.
  arXiv  Detail & Related papers  (2023-03-02T01:39:16Z)
• A Survey on Heterogeneous Federated Learning [12.395474890081232]
  Federated learning (FL) has been proposed to protect data privacy and assemble isolated data silos by cooperatively training models among organizations without breaching privacy and security. However, FL faces heterogeneous aspects, including data space, statistical, and system heterogeneity. We propose a precise taxonomy of heterogeneous FL settings for each type of heterogeneity according to the problem setting and learning objective.
  arXiv  Detail & Related papers  (2022-10-10T09:16:43Z)
• Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarks [65.34113135080105]
  We show that not only the issue of data heterogeneity in current setups is not necessarily a problem but also in fact it can be beneficial for the FL participants. Our observations are intuitive. Our code is available at https://github.com/MMorafah/FL-SC-NIID.
  arXiv  Detail & Related papers  (2022-09-30T17:15:19Z)
• Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training [60.892342868936865]
  Federated learning (FL) is a rapidly growing privacy-preserving collaborative machine learning paradigm. We propose a data heterogeneous-robust FL approach, FedGSP, to address this challenge. We show that FedGSP improves the accuracy by 3.7% on average compared with seven state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-01-31T03:15:28Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Non-IID data and Continual Learning processes in Federated Learning: A long road ahead [58.720142291102135]
  Federated Learning is a novel framework that allows multiple devices or institutions to train a machine learning model collaboratively while preserving their data private. In this work, we formally classify data statistical heterogeneity and review the most remarkable learning strategies that are able to face it. At the same time, we introduce approaches from other machine learning frameworks, such as Continual Learning, that also deal with data heterogeneity and could be easily adapted to the Federated Learning settings.
  arXiv  Detail & Related papers  (2021-11-26T09:57:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.