Heterogeneous Federated Learning: State-of-the-art and Research
Challenges
- URL: http://arxiv.org/abs/2307.10616v2
- Date: Fri, 8 Sep 2023 07:19:22 GMT
- Title: Heterogeneous Federated Learning: State-of-the-art and Research
Challenges
- Authors: Mang Ye, Xiuwen Fang, Bo Du, Pong C. Yuen, Dacheng Tao
- Abstract summary: Heterogeneous Federated Learning (HFL) is much more challenging and corresponding solutions are diverse and complex.
New advances in HFL are reviewed and a new taxonomy of existing HFL methods is proposed.
Several critical and promising future research directions in HFL are discussed.
- Score: 117.77132819796105
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated learning (FL) has drawn increasing attention owing to its potential
use in large-scale industrial applications. Existing federated learning works
mainly focus on model homogeneous settings. However, practical federated
learning typically faces the heterogeneity of data distributions, model
architectures, network environments, and hardware devices among participant
clients. Heterogeneous Federated Learning (HFL) is much more challenging, and
corresponding solutions are diverse and complex. Therefore, a systematic survey
on this topic about the research challenges and state-of-the-art is essential.
In this survey, we firstly summarize the various research challenges in HFL
from five aspects: statistical heterogeneity, model heterogeneity,
communication heterogeneity, device heterogeneity, and additional challenges.
In addition, recent advances in HFL are reviewed and a new taxonomy of existing
HFL methods is proposed with an in-depth analysis of their pros and cons. We
classify existing methods from three different levels according to the HFL
procedure: data-level, model-level, and server-level. Finally, several critical
and promising future research directions in HFL are discussed, which may
facilitate further developments in this field. A periodically updated
collection on HFL is available at https://github.com/marswhu/HFL_Survey.
Related papers
- Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey [67.48187503803847]
Vertical Federated Learning (VFL) is a privacy-preserving distributed learning paradigm.
Recent research has shown promising results addressing various challenges in VFL.
This survey offers a systematic overview of recent developments.
arXiv Detail & Related papers (2024-05-25T16:05:06Z) - Advances in Robust Federated Learning: Heterogeneity Considerations [25.261572089655264]
Key challenge is to efficiently train models across multiple clients with different data distributions, model structures, task objectives, computational capabilities, and communication resources.
In this paper, we first outline the basic concepts of heterogeneous federated learning.
We then summarize the research challenges in federated learning in terms of five aspects: data, model, task, device, and communication.
arXiv Detail & Related papers (2024-05-16T06:35:42Z) - SoK: Challenges and Opportunities in Federated Unlearning [32.0365189539138]
This SoK paper aims to take a deep look at the emphfederated unlearning literature, with the goal of identifying research trends and challenges in this emerging field.
arXiv Detail & Related papers (2024-03-04T19:35:08Z) - Federated Learning for Generalization, Robustness, Fairness: A Survey
and Benchmark [55.898771405172155]
Federated learning has emerged as a promising paradigm for privacy-preserving collaboration among different parties.
We provide a systematic overview of the important and recent developments of research on federated learning.
arXiv Detail & Related papers (2023-11-12T06:32:30Z) - Deep Equilibrium Models Meet Federated Learning [71.57324258813675]
This study explores the problem of Federated Learning (FL) by utilizing the Deep Equilibrium (DEQ) models instead of conventional deep learning networks.
We claim that incorporating DEQ models into the federated learning framework naturally addresses several open problems in FL.
To the best of our knowledge, this study is the first to establish a connection between DEQ models and federated learning.
arXiv Detail & Related papers (2023-05-29T22:51:40Z) - FS-Real: Towards Real-World Cross-Device Federated Learning [60.91678132132229]
Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data.
There is still a considerable gap between the flourishing FL research and real-world scenarios, mainly caused by the characteristics of heterogeneous devices and its scales.
We propose an efficient and scalable prototyping system for real-world cross-device FL, FS-Real.
arXiv Detail & Related papers (2023-03-23T15:37:17Z) - A Survey on Heterogeneous Federated Learning [12.395474890081232]
Federated learning (FL) has been proposed to protect data privacy and assemble isolated data silos by cooperatively training models among organizations without breaching privacy and security.
However, FL faces heterogeneous aspects, including data space, statistical, and system heterogeneity.
We propose a precise taxonomy of heterogeneous FL settings for each type of heterogeneity according to the problem setting and learning objective.
arXiv Detail & Related papers (2022-10-10T09:16:43Z) - Vertical Federated Learning: Challenges, Methodologies and Experiments [38.810479257782454]
vertical learning (VFL) is capable of constructing a hyper ML model by embracing sub-models from different clients.
In this paper, we discuss key challenges in VFL with effective solutions, and conduct experiments on real-life datasets.
arXiv Detail & Related papers (2022-02-09T06:56:41Z) - Heterogeneous Representation Learning: A Review [66.12816399765296]
Heterogeneous Representation Learning (HRL) brings some unique challenges.
We present a unified learning framework which is able to model most existing learning settings with the heterogeneous inputs.
We highlight the challenges that are less-touched in HRL and present future research directions.
arXiv Detail & Related papers (2020-04-28T05:12:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.