Hierarchical and Decentralised Federated Learning
- URL: http://arxiv.org/abs/2304.14982v1
- Date: Fri, 28 Apr 2023 17:06:50 GMT
- Title: Hierarchical and Decentralised Federated Learning
- Authors: Omer Rana, Theodoros Spyridopoulos, Nathaniel Hudson, Matt Baughman,
Kyle Chard, Ian Foster, Aftab Khan
- Abstract summary: Hierarchical Federated Learning extends the traditional FL process to enable more efficient model aggregation.
It can improve performance and reduce costs, whilst also enabling FL to be deployed in environments not well-suited to traditional FL.
H-FL will be crucial to future FL solutions as it can aggregate and distribute models at multiple levels to optimally serve the trade-off between locality dependence and global anomaly robustness.
- Score: 3.055801139718484
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Federated learning has shown enormous promise as a way of training ML models
in distributed environments while reducing communication costs and protecting
data privacy. However, the rise of complex cyber-physical systems, such as the
Internet-of-Things, presents new challenges that are not met with traditional
FL methods. Hierarchical Federated Learning extends the traditional FL process
to enable more efficient model aggregation based on application needs or
characteristics of the deployment environment (e.g., resource capabilities
and/or network connectivity). It illustrates the benefits of balancing
processing across the cloud-edge continuum. Hierarchical Federated Learning is
likely to be a key enabler for a wide range of applications, such as smart
farming and smart energy management, as it can improve performance and reduce
costs, whilst also enabling FL workflows to be deployed in environments that
are not well-suited to traditional FL. Model aggregation algorithms, software
frameworks, and infrastructures will need to be designed and implemented to
make such solutions accessible to researchers and engineers across a growing
set of domains.
H-FL also introduces a number of new challenges. For instance, there are
implicit infrastructural challenges. There is also a trade-off between having
generalised models and personalised models. If there exist geographical
patterns for data (e.g., soil conditions in a smart farm likely are related to
the geography of the region itself), then it is crucial that models used
locally can consider their own locality in addition to a globally-learned
model. H-FL will be crucial to future FL solutions as it can aggregate and
distribute models at multiple levels to optimally serve the trade-off between
locality dependence and global anomaly robustness.
Related papers
- Federated Learning for Non-factorizable Models using Deep Generative Prior Approximations [0.6990493129893112]
Federated learning (FL) allows for collaborative model training across decentralized clients while preserving privacy by avoiding data sharing.
We introduce the Structured Independence via deep Generative Model Approximation ( SIGMA) prior which enables FL for non-factorizable models across clients.
We demonstrate the SIGMA prior's effectiveness on synthetic data and showcase its utility in a real-world example of FL for spatial data, using a conditional autoregressive prior to model spatial dependence across Australia.
arXiv Detail & Related papers (2024-05-25T04:44:06Z) - A Survey on Efficient Federated Learning Methods for Foundation Model
Training [66.19763977571114]
Federated Learning (FL) has become an established technique to facilitate privacy-preserving collaborative training across a multitude of clients.
In the wake of Foundation Models (FM), the reality is different for many deep learning applications.
We discuss the benefits and drawbacks of parameter-efficient fine-tuning (PEFT) for FL applications.
arXiv Detail & Related papers (2024-01-09T10:22:23Z) - Deep Equilibrium Models Meet Federated Learning [71.57324258813675]
This study explores the problem of Federated Learning (FL) by utilizing the Deep Equilibrium (DEQ) models instead of conventional deep learning networks.
We claim that incorporating DEQ models into the federated learning framework naturally addresses several open problems in FL.
To the best of our knowledge, this study is the first to establish a connection between DEQ models and federated learning.
arXiv Detail & Related papers (2023-05-29T22:51:40Z) - Vertical Federated Learning over Cloud-RAN: Convergence Analysis and
System Optimization [82.12796238714589]
We propose a novel cloud radio access network (Cloud-RAN) based vertical FL system to enable fast and accurate model aggregation.
We characterize the convergence behavior of the vertical FL algorithm considering both uplink and downlink transmissions.
We establish a system optimization framework by joint transceiver and fronthaul quantization design, for which successive convex approximation and alternate convex search based system optimization algorithms are developed.
arXiv Detail & Related papers (2023-05-04T09:26:03Z) - Fed-FSNet: Mitigating Non-I.I.D. Federated Learning via Fuzzy
Synthesizing Network [19.23943687834319]
Federated learning (FL) has emerged as a promising privacy-preserving distributed machine learning framework.
We propose a novel FL training framework, dubbed Fed-FSNet, using a properly designed Fuzzy Synthesizing Network (FSNet) to mitigate the Non-I.I.D. at-the-source issue.
arXiv Detail & Related papers (2022-08-21T18:40:51Z) - Introducing Federated Learning into Internet of Things ecosystems --
preliminary considerations [0.31402652384742363]
Federated learning (FL) was proposed to facilitate the training of models in a distributed environment.
It supports the protection of (local) data privacy and uses local resources for model training.
arXiv Detail & Related papers (2022-07-15T18:48:57Z) - Efficient Split-Mix Federated Learning for On-Demand and In-Situ
Customization [107.72786199113183]
Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data.
In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
arXiv Detail & Related papers (2022-03-18T04:58:34Z) - Fine-tuning Global Model via Data-Free Knowledge Distillation for
Non-IID Federated Learning [86.59588262014456]
Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint.
We propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG)
Our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
arXiv Detail & Related papers (2022-03-17T11:18:17Z) - Personalized Federated Learning with Clustered Generalization [16.178571176116073]
We study the recent emerging personalized learning (PFL) that aims at dealing with the challenging problem of Non-I.I.D. data in the learning setting.
Key difference between PFL and conventional FL methods in the training target.
We propose a novel concept called clustered generalization to handle the challenge of statistical heterogeneity in FL.
arXiv Detail & Related papers (2021-06-24T14:17:00Z) - Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn.
We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
arXiv Detail & Related papers (2020-12-01T11:46:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.