MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcare

• URL: http://arxiv.org/abs/2206.08516v3
• Date: Sun, 9 Jul 2023 12:46:01 GMT
• Title: MetaFed: Federated Learning among Federations with Cyclic Knowledge Distillation for Personalized Healthcare
• Abstract summary: Federated learning has attracted increasing attention to building models without accessing the raw user data. We propose a novel framework called MetaFed to facilitate trustworthy FL between different federations.
• Score: 22.248585185394347
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated learning has attracted increasing attention to building models without accessing the raw user data, especially in healthcare. In real applications, different federations can seldom work together due to possible reasons such as data heterogeneity and distrust/inexistence of the central server. In this paper, we propose a novel framework called MetaFed to facilitate trustworthy FL between different federations. MetaFed obtains a personalized model for each federation without a central server via the proposed Cyclic Knowledge Distillation. Specifically, MetaFed treats each federation as a meta distribution and aggregates knowledge of each federation in a cyclic manner. The training is split into two parts: common knowledge accumulation and personalization. Comprehensive experiments on three benchmarks demonstrate that MetaFed without a server achieves better accuracy compared to state-of-the-art methods (e.g., 10%+ accuracy improvement compared to the baseline for PAMAP2) with fewer communication costs.

Related papers

• Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion [59.54067771781552]
  We propose a framework named MMFeD3-HidE for addressing multimodal uncertain unavailability and multimodal client heterogeneity challenges of FedMKGC.<n>We propose a FedMKGC benchmark for a comprehensive evaluation, consisting of a general FedMKGC backbone named MMFedE, datasets with heterogeneous multimodal information, and three groups of constructed baselines.
  arXiv  Detail & Related papers  (2025-06-27T09:32:58Z)
• Federated brain tumor segmentation: an extensive benchmark [2.515027627030043]
  We propose an extensive benchmark of federated learning algorithms from all three classes on this task. We show that some methods from each category can bring a slight performance improvement and potentially limit the final model(s) bias toward the predominant data distribution of the federation.
  arXiv  Detail & Related papers  (2024-10-07T09:32:19Z)
• Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Data [6.024145412139383]
  Federated learning is an emerging distributed machine learning framework aiming at protecting data privacy. Data in different clients contain both common knowledge and personalized knowledge. We develop a novel personalized federated learning framework for heterogeneous data, which we refer to as FedSplit.
  arXiv  Detail & Related papers  (2023-12-07T13:05:47Z)
• Combating Exacerbated Heterogeneity for Robust Models in Federated Learning [91.88122934924435]
  Combination of adversarial training and federated learning can lead to the undesired robustness deterioration. We propose a novel framework called Slack Federated Adversarial Training (SFAT) We verify the rationality and effectiveness of SFAT on various benchmarked and real-world datasets.
  arXiv  Detail & Related papers  (2023-03-01T06:16:15Z)
• FedABC: Targeting Fair Competition in Personalized Federated Learning [76.9646903596757]
  Federated learning aims to collaboratively train models without accessing their client's local private data. We propose a novel and generic PFL framework termed Federated Averaging via Binary Classification, dubbed FedABC. In particular, we adopt the one-vs-all'' training strategy in each client to alleviate the unfair competition between classes.
  arXiv  Detail & Related papers  (2023-02-15T03:42:59Z)
• Meta Knowledge Condensation for Federated Learning [65.20774786251683]
  Existing federated learning paradigms usually extensively exchange distributed models at a central solver to achieve a more powerful model. This would incur severe communication burden between a server and multiple clients especially when data distributions are heterogeneous. Unlike existing paradigms, we introduce an alternative perspective to significantly decrease the communication cost in federate learning.
  arXiv  Detail & Related papers  (2022-09-29T15:07:37Z)
• Motley: Benchmarking Heterogeneity and Personalization in Federated Learning [20.66924459164993]
  Motley is a benchmark for personalized federated learning. It consists of a suite of cross-device and cross-silo federated datasets from varied problem domains. We establish baselines on the benchmark by comparing a number of representative personalized federated learning methods.
  arXiv  Detail & Related papers  (2022-06-18T18:18:49Z)
• FedFormer: Contextual Federation with Attention in Reinforcement Learning [21.05134144161283]
  FedFormer is a novel federation strategy that utilizes Transformer Attention to contextually aggregate embeddings. We evaluate our methods on the Meta-World environment and find that our approach yields significant improvements over FedAvg and non-federated Soft Actor Critique single agent methods.
  arXiv  Detail & Related papers  (2022-05-27T01:19:22Z)
• Practical One-Shot Federated Learning for Cross-Silo Setting [114.76232507580067]
  One-shot federated learning is a promising approach to make federated learning applicable in cross-silo setting. We propose a practical one-shot federated learning algorithm named FedKT. By utilizing the knowledge transfer technique, FedKT can be applied to any classification models and can flexibly achieve differential privacy guarantees.
  arXiv  Detail & Related papers  (2020-10-02T14:09:10Z)
• WAFFLe: Weight Anonymized Factorization for Federated Learning [88.44939168851721]
  In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. We propose Weight Anonymized Factorization for Federated Learning (WAFFLe), an approach that combines the Indian Buffet Process with a shared dictionary of weight factors for neural networks.
  arXiv  Detail & Related papers  (2020-08-13T04:26:31Z)
• Multi-Center Federated Learning [62.57229809407692]
  This paper proposes a novel multi-center aggregation mechanism for federated learning. It learns multiple global models from the non-IID user data and simultaneously derives the optimal matching between users and centers. Our experimental results on benchmark datasets show that our method outperforms several popular federated learning methods.
  arXiv  Detail & Related papers  (2020-05-03T09:14:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.