Heterogeneous Federated Learning: State-of-the-art and Research Challenges

• URL: http://arxiv.org/abs/2307.10616v2
• Date: Fri, 8 Sep 2023 07:19:22 GMT
• Title: Heterogeneous Federated Learning: State-of-the-art and Research Challenges
• Authors: Mang Ye, Xiuwen Fang, Bo Du, Pong C. Yuen, Dacheng Tao
• Abstract summary: Heterogeneous Federated Learning (HFL) is much more challenging and corresponding solutions are diverse and complex. New advances in HFL are reviewed and a new taxonomy of existing HFL methods is proposed. Several critical and promising future research directions in HFL are discussed.
• Score: 117.77132819796105
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) has drawn increasing attention owing to its potential use in large-scale industrial applications. Existing federated learning works mainly focus on model homogeneous settings. However, practical federated learning typically faces the heterogeneity of data distributions, model architectures, network environments, and hardware devices among participant clients. Heterogeneous Federated Learning (HFL) is much more challenging, and corresponding solutions are diverse and complex. Therefore, a systematic survey on this topic about the research challenges and state-of-the-art is essential. In this survey, we firstly summarize the various research challenges in HFL from five aspects: statistical heterogeneity, model heterogeneity, communication heterogeneity, device heterogeneity, and additional challenges. In addition, recent advances in HFL are reviewed and a new taxonomy of existing HFL methods is proposed with an in-depth analysis of their pros and cons. We classify existing methods from three different levels according to the HFL procedure: data-level, model-level, and server-level. Finally, several critical and promising future research directions in HFL are discussed, which may facilitate further developments in this field. A periodically updated collection on HFL is available at https://github.com/marswhu/HFL_Survey.

Related papers

• One-shot Federated Learning Methods: A Practical Guide [23.737787001337082]
  One-shot Federated Learning (OFL) is a distributed machine learning paradigm that constrains client-server communication to a single round. This paper presents a systematic analysis of the challenges faced by OFL and thoroughly reviews the current methods.
  arXiv  Detail & Related papers  (2025-02-13T09:26:44Z)
• Comparative Evaluation of Clustered Federated Learning Methods [0.5242869847419834]
  Clustered Federated Learning (CFL) aims to partition clients into groups where the distribution are homogeneous. In this paper, we explore the performance of two state-of-theart CFL algorithms with respect to a proposed taxonomy of data heterogeneities in federated learning (FL) Our objective is to provide a clearer understanding of the relationship between CFL performances and data heterogeneous scenarios.
  arXiv  Detail & Related papers  (2024-10-18T07:01:56Z)
• Federated Large Language Models: Current Progress and Future Directions [63.68614548512534]
  This paper surveys Federated learning for LLMs (FedLLM), highlighting recent advances and future directions. We focus on two key aspects: fine-tuning and prompt learning in a federated setting, discussing existing work and associated research challenges.
  arXiv  Detail & Related papers  (2024-09-24T04:14:33Z)
• Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey [67.48187503803847]
  Vertical Federated Learning (VFL) is a privacy-preserving distributed learning paradigm. Recent research has shown promising results addressing various challenges in VFL. This survey offers a systematic overview of recent developments.
  arXiv  Detail & Related papers  (2024-05-25T16:05:06Z)
• Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark [55.898771405172155]
  Federated learning has emerged as a promising paradigm for privacy-preserving collaboration among different parties. We provide a systematic overview of the important and recent developments of research on federated learning.
  arXiv  Detail & Related papers  (2023-11-12T06:32:30Z)
• FS-Real: Towards Real-World Cross-Device Federated Learning [60.91678132132229]
  Federated Learning (FL) aims to train high-quality models in collaboration with distributed clients while not uploading their local data. There is still a considerable gap between the flourishing FL research and real-world scenarios, mainly caused by the characteristics of heterogeneous devices and its scales. We propose an efficient and scalable prototyping system for real-world cross-device FL, FS-Real.
  arXiv  Detail & Related papers  (2023-03-23T15:37:17Z)
• A Survey on Heterogeneous Federated Learning [12.395474890081232]
  Federated learning (FL) has been proposed to protect data privacy and assemble isolated data silos by cooperatively training models among organizations without breaching privacy and security. However, FL faces heterogeneous aspects, including data space, statistical, and system heterogeneity. We propose a precise taxonomy of heterogeneous FL settings for each type of heterogeneity according to the problem setting and learning objective.
  arXiv  Detail & Related papers  (2022-10-10T09:16:43Z)
• Vertical Federated Learning: Challenges, Methodologies and Experiments [34.4865409422585]
  vertical learning (VFL) is capable of constructing a hyper ML model by embracing sub-models from different clients. In this paper, we discuss key challenges in VFL with effective solutions, and conduct experiments on real-life datasets.
  arXiv  Detail & Related papers  (2022-02-09T06:56:41Z)
• Heterogeneous Representation Learning: A Review [66.12816399765296]
  Heterogeneous Representation Learning (HRL) brings some unique challenges. We present a unified learning framework which is able to model most existing learning settings with the heterogeneous inputs. We highlight the challenges that are less-touched in HRL and present future research directions.
  arXiv  Detail & Related papers  (2020-04-28T05:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.