Cross-Silo Federated Learning: Challenges and Opportunities
- URL: http://arxiv.org/abs/2206.12949v1
- Date: Sun, 26 Jun 2022 19:49:41 GMT
- Title: Cross-Silo Federated Learning: Challenges and Opportunities
- Authors: Chao Huang, Jianwei Huang, Xin Liu
- Abstract summary: Federated learning (FL) enables the training of machine learning models from multiple clients while keeping the data distributed and private.
Based on the participating clients and the model training scale, federated learning can be classified into two types: cross-device FL where clients are typically mobile devices and the client number can reach up to a scale of millions; cross-silo FL where clients are organizations or companies and the client number is usually small (e.g., within a hundred)
- Score: 30.351077030186104
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Federated learning (FL) is an emerging technology that enables the training
of machine learning models from multiple clients while keeping the data
distributed and private. Based on the participating clients and the model
training scale, federated learning can be classified into two types:
cross-device FL where clients are typically mobile devices and the client
number can reach up to a scale of millions; cross-silo FL where clients are
organizations or companies and the client number is usually small (e.g., within
a hundred). While existing studies mainly focus on cross-device FL, this paper
aims to provide an overview of the cross-silo FL. More specifically, we first
discuss applications of cross-silo FL and outline its major challenges. We then
provide a systematic overview of the existing approaches to the challenges in
cross-silo FL by focusing on their connections and differences to cross-device
FL. Finally, we discuss future directions and open issues that merit research
efforts from the community.
Related papers
- Federated Distillation: A Survey [73.08661634882195]
Federated Learning (FL) seeks to train a model collaboratively without sharing private training data from individual clients.
The integration of knowledge distillation into FL has been proposed, forming what is known as Federated Distillation (FD)
FD enables more flexible knowledge transfer between clients and the server, surpassing the mere sharing of model parameters.
arXiv Detail & Related papers (2024-04-02T03:42:18Z) - A Survey on Efficient Federated Learning Methods for Foundation Model
Training [66.19763977571114]
Federated Learning (FL) has become an established technique to facilitate privacy-preserving collaborative training across a multitude of clients.
In the wake of Foundation Models (FM), the reality is different for many deep learning applications.
We discuss the benefits and drawbacks of parameter-efficient fine-tuning (PEFT) for FL applications.
arXiv Detail & Related papers (2024-01-09T10:22:23Z) - A Survey on Decentralized Federated Learning [0.709016563801433]
In recent years, federated learning has become a popular paradigm for training distributed, large-scale, and privacy-preserving machine learning (ML) systems.
In a typical FL system, the central server acts only as an orchestrator; it iteratively gathers and aggregates all the local models trained by each client on its private data until convergence.
One of the most critical challenges is to overcome the centralized orchestration of the classical FL client-server architecture.
Decentralized FL solutions have emerged where all FL clients cooperate and communicate without a central server.
arXiv Detail & Related papers (2023-08-08T22:07:15Z) - FL Games: A Federated Learning Framework for Distribution Shifts [71.98708418753786]
Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server.
We propose FL GAMES, a game-theoretic framework for federated learning that learns causal features that are invariant across clients.
arXiv Detail & Related papers (2022-10-31T22:59:03Z) - FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in
Realistic Healthcare Settings [51.09574369310246]
Federated Learning (FL) is a novel approach enabling several clients holding sensitive data to collaboratively train machine learning models.
We propose a novel cross-silo dataset suite focused on healthcare, FLamby, to bridge the gap between theory and practice of cross-silo FL.
Our flexible and modular suite allows researchers to easily download datasets, reproduce results and re-use the different components for their research.
arXiv Detail & Related papers (2022-10-10T12:17:30Z) - Federated Learning from Pre-Trained Models: A Contrastive Learning
Approach [43.893267526525904]
Federated Learning (FL) is a machine learning paradigm that allows decentralized clients to learn collaboratively without sharing their private data.
Excessive computation and communication demands pose challenges to current FL frameworks.
We propose a lightweight framework where clients jointly learn to fuse the representations generated by multiple fixed pre-trained models.
arXiv Detail & Related papers (2022-09-21T03:16:57Z) - A Survey on Participant Selection for Federated Learning in Mobile
Networks [47.88372677863646]
Federated Learning (FL) is an efficient distributed machine learning paradigm that employs private datasets in a privacy-preserving manner.
Due to limited communication bandwidth and unstable availability of such devices in a mobile network, only a fraction of end devices can be selected in each round.
arXiv Detail & Related papers (2022-07-08T04:22:48Z) - FederatedScope: A Comprehensive and Flexible Federated Learning Platform
via Message Passing [63.87056362712879]
We propose a novel and comprehensive federated learning platform, named FederatedScope, which is based on a message-oriented framework.
Compared to the procedural framework, the proposed message-oriented framework is more flexible to express heterogeneous message exchange.
We conduct a series of experiments on the provided easy-to-use and comprehensive FL benchmarks to validate the correctness and efficiency of FederatedScope.
arXiv Detail & Related papers (2022-04-11T11:24:21Z) - Federated Learning for Resource-Constrained IoT Devices: Panoramas and
State-of-the-art [12.129978716326676]
We introduce some recently implemented real-life applications of Federated Learning.
In large-scale networks, there may be clients with varying computational resource capabilities.
We highlight future directions in the FL area concerning resource-constrained devices.
arXiv Detail & Related papers (2020-02-25T01:03:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.