A Survey on Heterogeneous Federated Learning
- URL: http://arxiv.org/abs/2210.04505v1
- Date: Mon, 10 Oct 2022 09:16:43 GMT
- Title: A Survey on Heterogeneous Federated Learning
- Authors: Dashan Gao, Xin Yao, Qiang Yang
- Abstract summary: Federated learning (FL) has been proposed to protect data privacy and assemble isolated data silos by cooperatively training models among organizations without breaching privacy and security.
However, FL faces heterogeneous aspects, including data space, statistical, and system heterogeneity.
We propose a precise taxonomy of heterogeneous FL settings for each type of heterogeneity according to the problem setting and learning objective.
- Score: 12.395474890081232
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Federated learning (FL) has been proposed to protect data privacy and
virtually assemble the isolated data silos by cooperatively training models
among organizations without breaching privacy and security. However, FL faces
heterogeneity from various aspects, including data space, statistical, and
system heterogeneity. For example, collaborative organizations without conflict
of interest often come from different areas and have heterogeneous data from
different feature spaces. Participants may also want to train heterogeneous
personalized local models due to non-IID and imbalanced data distribution and
various resource-constrained devices. Therefore, heterogeneous FL is proposed
to address the problem of heterogeneity in FL. In this survey, we
comprehensively investigate the domain of heterogeneous FL in terms of data
space, statistical, system, and model heterogeneity. We first give an overview
of FL, including its definition and categorization. Then, We propose a precise
taxonomy of heterogeneous FL settings for each type of heterogeneity according
to the problem setting and learning objective. We also investigate the transfer
learning methodologies to tackle the heterogeneity in FL. We further present
the applications of heterogeneous FL. Finally, we highlight the challenges and
opportunities and envision promising future research directions toward new
framework design and trustworthy approaches.
Related papers
- Synthetic Data Aided Federated Learning Using Foundation Models [4.666380225768727]
We propose Differentially Private Synthetic Data Aided Federated Learning Using Foundation Models (DPSDA-FL)
Our experimental results have shown that DPSDA-FL can improve class recall and classification accuracy of the global model by up to 26% and 9%, respectively, in FL with Non-IID issues.
arXiv Detail & Related papers (2024-07-06T20:31:43Z) - On the Impact of Data Heterogeneity in Federated Learning Environments with Application to Healthcare Networks [3.6904315184369154]
Federated Learning (FL) allows privacy-sensitive applications to leverage their dataset for a global model construction without any disclosure of the information.
One of those domains is healthcare, where groups of silos collaborate in order to generate a global predictor with improved accuracy and generalization.
This paper presents a comprehensive exploration of the mathematical formalization and taxonomy of heterogeneity within FL environments, focusing on the intricacies of medical data.
arXiv Detail & Related papers (2024-04-29T09:05:01Z) - Federated Learning Empowered by Generative Content [55.576885852501775]
Federated learning (FL) enables leveraging distributed private data for model training in a privacy-preserving way.
We propose a novel FL framework termed FedGC, designed to mitigate data heterogeneity issues by diversifying private data with generative content.
We conduct a systematic empirical study on FedGC, covering diverse baselines, datasets, scenarios, and modalities.
arXiv Detail & Related papers (2023-12-10T07:38:56Z) - FedSym: Unleashing the Power of Entropy for Benchmarking the Algorithms
for Federated Learning [1.4656078321003647]
Federated learning (FL) is a decentralized machine learning approach where independent learners process data privately.
We study the currently popular data partitioning techniques and visualize their main disadvantages.
We propose a method that leverages entropy and symmetry to construct 'the most challenging' and controllable data distributions.
arXiv Detail & Related papers (2023-10-11T18:39:08Z) - Generalizable Heterogeneous Federated Cross-Correlation and Instance
Similarity Learning [60.058083574671834]
This paper presents a novel FCCL+, federated correlation and similarity learning with non-target distillation.
For heterogeneous issue, we leverage irrelevant unlabeled public data for communication.
For catastrophic forgetting in local updating stage, FCCL+ introduces Federated Non Target Distillation.
arXiv Detail & Related papers (2023-09-28T09:32:27Z) - PFL-GAN: When Client Heterogeneity Meets Generative Models in
Personalized Federated Learning [55.930403371398114]
We propose a novel generative adversarial network (GAN) sharing and aggregation strategy for personalized learning (PFL)
PFL-GAN addresses the client heterogeneity in different scenarios. More specially, we first learn the similarity among clients and then develop an weighted collaborative data aggregation.
The empirical results through the rigorous experimentation on several well-known datasets demonstrate the effectiveness of PFL-GAN.
arXiv Detail & Related papers (2023-08-23T22:38:35Z) - Federated Learning for Data and Model Heterogeneity in Medical Imaging [19.0931609571649]
Federated Learning (FL) is an evolving machine learning method in which multiple clients participate in collaborative learning without sharing their data with each other and the central server.
In real-world applications such as hospitals and industries, FL counters the challenges of data Heterogeneity and Model Heterogeneity.
We propose a method, MDH-FL (Exploiting Model and Data Heterogeneity in FL), to solve such problems.
arXiv Detail & Related papers (2023-07-31T21:08:45Z) - Heterogeneous Federated Learning: State-of-the-art and Research
Challenges [117.77132819796105]
Heterogeneous Federated Learning (HFL) is much more challenging and corresponding solutions are diverse and complex.
New advances in HFL are reviewed and a new taxonomy of existing HFL methods is proposed.
Several critical and promising future research directions in HFL are discussed.
arXiv Detail & Related papers (2023-07-20T06:32:14Z) - Completely Heterogeneous Federated Learning [16.426770356031636]
Federated learning (FL) faces three major difficulties: cross-domain, heterogeneous models, and non-i.i.d. labels scenarios.
We propose the challenging "completely heterogeneous" scenario in FL, which refers to that each client will not expose any private information.
We then devise an FL framework based on parameter decoupling and data-free knowledge distillation to solve the problem.
arXiv Detail & Related papers (2022-10-28T03:20:10Z) - Local Learning Matters: Rethinking Data Heterogeneity in Federated
Learning [61.488646649045215]
Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
arXiv Detail & Related papers (2021-11-28T19:03:39Z) - FedH2L: Federated Learning with Model and Statistical Heterogeneity [75.61234545520611]
Federated learning (FL) enables distributed participants to collectively learn a strong global model without sacrificing their individual data privacy.
We introduce FedH2L, which is agnostic to both the model architecture and robust to different data distributions across participants.
In contrast to approaches sharing parameters or gradients, FedH2L relies on mutual distillation, exchanging only posteriors on a shared seed set between participants in a decentralized manner.
arXiv Detail & Related papers (2021-01-27T10:10:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.