FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation

• URL: http://arxiv.org/abs/2210.08285v2
• Date: Thu, 4 Jul 2024 17:58:58 GMT
• Title: FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation
• Authors: Ming Hu, Peiheng Zhou, Zhihao Yue, Zhiwei Ling, Yihao Huang, Anran Li, Yang Liu, Xiang Lian, Mingsong Chen,
• Abstract summary: Federated Learning (FL) has attracted increasing attention to deal with data silo problems without compromising user privacy. We present an efficient FL framework named FedCross, which uses a novel multi-to-multi FL training scheme based on our proposed multi-model cross-aggregation approach. We show that FedCross can significantly improve FL accuracy within both IID and non-IID scenarios without causing additional communication overhead.
• Score: 16.019513233021435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As a promising distributed machine learning paradigm, Federated Learning (FL) has attracted increasing attention to deal with data silo problems without compromising user privacy. By adopting the classic one-to-multi training scheme (i.e., FedAvg), where the cloud server dispatches one single global model to multiple involved clients, conventional FL methods can achieve collaborative model training without data sharing. However, since only one global model cannot always accommodate all the incompatible convergence directions of local models, existing FL approaches greatly suffer from inferior classification accuracy. To address this issue, we present an efficient FL framework named FedCross, which uses a novel multi-to-multi FL training scheme based on our proposed multi-model cross-aggregation approach. Unlike traditional FL methods, in each round of FL training, FedCross uses multiple middleware models to conduct weighted fusion individually. Since the middleware models used by FedCross can quickly converge into the same flat valley in terms of loss landscapes, the generated global model can achieve a well-generalization. Experimental results on various well-known datasets show that, compared with state-of-the-art FL methods, FedCross can significantly improve FL accuracy within both IID and non-IID scenarios without causing additional communication overhead.

Related papers

• Tunable Soft Prompts are Messengers in Federated Learning [55.924749085481544]
  Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources. The lack of model privacy protection in FL becomes an unneglectable challenge. We propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts.
  arXiv  Detail & Related papers  (2023-11-12T11:01:10Z)
• Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training [67.67045085186797]
  Almost all existing systems have to face large communication burdens if the central FL server fails. It personalizes the "right" in the deep models by alternately updating the shared and personal parameters. To further promote the shared parameters aggregation process, we propose DFed integrating the local Sharpness Miniization.
  arXiv  Detail & Related papers  (2023-05-24T13:52:18Z)
• FedMR: Fedreated Learning via Model Recombination [7.404225808071622]
  Federated Learning (FL) enables global model training across clients without compromising their confidential local data. Existing FL methods rely on Federated Averaging (FedAvg)-based aggregation. This paper proposes a novel and effective FL paradigm named FedMR (Federating Model Recombination)
  arXiv  Detail & Related papers  (2022-08-16T11:30:19Z)
• Multi-Model Federated Learning with Provable Guarantees [19.470024548995717]
  Federated Learning (FL) is a variant of distributed learning where devices collaborate to learn a model without sharing their data with the central server or each other. We refer to the process of multiple independent clients simultaneously in a federated setting using a common pool of clients as a multi-model edge FL.
  arXiv  Detail & Related papers  (2022-07-09T19:47:52Z)
• Federated Cross Learning for Medical Image Segmentation [23.075410916203005]
  Federated learning (FL) can collaboratively train deep learning models using isolated patient data owned by different hospitals for various clinical applications. A major problem of FL is its performance degradation when dealing with data that are not independently and identically distributed (non-iid)
  arXiv  Detail & Related papers  (2022-04-05T18:55:02Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)
• Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning [86.59588262014456]
  Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint. We propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG) Our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
  arXiv  Detail & Related papers  (2022-03-17T11:18:17Z)
• FedCAT: Towards Accurate Federated Learning via Device Concatenation [4.416919766772866]
  Federated Learning (FL) enables all the involved devices to train a global model collaboratively without exposing their local data privacy. For non-IID scenarios, the classification accuracy of FL models decreases drastically due to the weight divergence caused by data heterogeneity. We introduce a novel FL approach named Fed-Cat that can achieve high model accuracy based on our proposed device selection strategy and device concatenation-based local training method.
  arXiv  Detail & Related papers  (2022-02-23T10:08:43Z)
• Multi-Center Federated Learning [62.32725938999433]
  Federated learning (FL) can protect data privacy in distributed learning. It merely collects local gradients from users without access to their data. We propose a novel multi-center aggregation mechanism.
  arXiv  Detail & Related papers  (2021-08-19T12:20:31Z)
• Personalized Federated Learning with Clustered Generalization [16.178571176116073]
  We study the recent emerging personalized learning (PFL) that aims at dealing with the challenging problem of Non-I.I.D. data in the learning setting. Key difference between PFL and conventional FL methods in the training target. We propose a novel concept called clustered generalization to handle the challenge of statistical heterogeneity in FL.
  arXiv  Detail & Related papers  (2021-06-24T14:17:00Z)
• A Bayesian Federated Learning Framework with Online Laplace Approximation [144.7345013348257]
  Federated learning allows multiple clients to collaboratively learn a globally shared model. We propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side. We achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.
  arXiv  Detail & Related papers  (2021-02-03T08:36:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.