Advances in Robust Federated Learning: Heterogeneity Considerations

• URL: http://arxiv.org/abs/2405.09839v1
• Date: Thu, 16 May 2024 06:35:42 GMT
• Title: Advances in Robust Federated Learning: Heterogeneity Considerations
• Authors: Chuan Chen, Tianchi Liao, Xiaojun Deng, Zihou Wu, Sheng Huang, Zibin Zheng,
• Abstract summary: Key challenge is to efficiently train models across multiple clients with different data distributions, model structures, task objectives, computational capabilities, and communication resources. In this paper, we first outline the basic concepts of heterogeneous federated learning. We then summarize the research challenges in federated learning in terms of five aspects: data, model, task, device, and communication.
• Score: 25.261572089655264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the field of heterogeneous federated learning (FL), the key challenge is to efficiently and collaboratively train models across multiple clients with different data distributions, model structures, task objectives, computational capabilities, and communication resources. This diversity leads to significant heterogeneity, which increases the complexity of model training. In this paper, we first outline the basic concepts of heterogeneous federated learning and summarize the research challenges in federated learning in terms of five aspects: data, model, task, device, and communication. In addition, we explore how existing state-of-the-art approaches cope with the heterogeneity of federated learning, and categorize and review these approaches at three different levels: data-level, model-level, and architecture-level. Subsequently, the paper extensively discusses privacy-preserving strategies in heterogeneous federated learning environments. Finally, the paper discusses current open issues and directions for future research, aiming to promote the further development of heterogeneous federated learning.

Related papers

• Cross-Training with Multi-View Knowledge Fusion for Heterogenous Federated Learning [13.796783869133531]
  This paper presents a novel approach that enhances federated learning through a cross-training scheme incorporating multi-view information. Specifically, the proposed method, termed FedCT, includes three main modules, where the consistency-aware knowledge broadcasting module aims to optimize model assignment strategies. The multi-view knowledge-guided representation learning module leverages fused knowledge from both global and local views to enhance the preservation of local knowledge before and after model exchange. The mixup-based feature augmentation module aggregates rich information to further increase the diversity of feature spaces, which enables the model to better discriminate complex samples.
  arXiv  Detail & Related papers  (2024-05-30T13:27:30Z)
• Advances and Open Challenges in Federated Learning with Foundation Models [34.389498604151946]
  The integration of Foundation Models (FMs) with Federated Learning (FL) presents a transformative paradigm in Artificial Intelligence (AI) This paper provides a comprehensive survey of the emerging field of Federated Foundation Models (FedFM)
  arXiv  Detail & Related papers  (2024-04-23T09:44:58Z)
• Heterogeneous Contrastive Learning for Foundation Models and Beyond [73.74745053250619]
  In the era of big data and Artificial Intelligence, an emerging paradigm is to utilize contrastive self-supervised learning to model large-scale heterogeneous data. This survey critically evaluates the current landscape of heterogeneous contrastive learning for foundation models.
  arXiv  Detail & Related papers  (2024-03-30T02:55:49Z)
• Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark [55.898771405172155]
  Federated learning has emerged as a promising paradigm for privacy-preserving collaboration among different parties. We provide a systematic overview of the important and recent developments of research on federated learning.
  arXiv  Detail & Related papers  (2023-11-12T06:32:30Z)
• Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learning [60.058083574671834]
  This paper presents a novel FCCL+, federated correlation and similarity learning with non-target distillation. For heterogeneous issue, we leverage irrelevant unlabeled public data for communication. For catastrophic forgetting in local updating stage, FCCL+ introduces Federated Non Target Distillation.
  arXiv  Detail & Related papers  (2023-09-28T09:32:27Z)
• Heterogeneous Federated Learning: State-of-the-art and Research Challenges [117.77132819796105]
  Heterogeneous Federated Learning (HFL) is much more challenging and corresponding solutions are diverse and complex. New advances in HFL are reviewed and a new taxonomy of existing HFL methods is proposed. Several critical and promising future research directions in HFL are discussed.
  arXiv  Detail & Related papers  (2023-07-20T06:32:14Z)
• Federated Hetero-Task Learning [42.985155807178685]
  We present B-FHTL, a federated hetero-task learning benchmark consisted of simulation dataset, FL protocols and a unified evaluation mechanism. To ensure fair comparison among different FL algorithms, B-FHTL builds in a full suite of FL protocols. We compare the FL algorithms in fields of federated multi-task learning, federated personalization and federated meta learning within B-FHTL.
  arXiv  Detail & Related papers  (2022-06-07T16:43:09Z)
• Non-IID data and Continual Learning processes in Federated Learning: A long road ahead [58.720142291102135]
  Federated Learning is a novel framework that allows multiple devices or institutions to train a machine learning model collaboratively while preserving their data private. In this work, we formally classify data statistical heterogeneity and review the most remarkable learning strategies that are able to face it. At the same time, we introduce approaches from other machine learning frameworks, such as Continual Learning, that also deal with data heterogeneity and could be easily adapted to the Federated Learning settings.
  arXiv  Detail & Related papers  (2021-11-26T09:57:11Z)
• Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning [65.06445195580622]
  Federated learning is a new paradigm that decouples data collection and model training via multi-party computation and model aggregation. We conduct a focused survey of federated learning in conjunction with other learning algorithms.
  arXiv  Detail & Related papers  (2021-02-25T15:18:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.