Timely Asynchronous Hierarchical Federated Learning: Age of Convergence
- URL: http://arxiv.org/abs/2306.12400v1
- Date: Wed, 21 Jun 2023 17:39:16 GMT
- Title: Timely Asynchronous Hierarchical Federated Learning: Age of Convergence
- Authors: Purbesh Mitra and Sennur Ulukus
- Abstract summary: We consider an asynchronous hierarchical federated learning setting with a client-edge-cloud framework.
The clients exchange the trained parameters with their corresponding edge servers, which update the locally aggregated model.
The goal of each client is to converge to the global model, while maintaining timeliness of the clients.
- Score: 59.96266198512243
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We consider an asynchronous hierarchical federated learning (AHFL) setting
with a client-edge-cloud framework. The clients exchange the trained parameters
with their corresponding edge servers, which update the locally aggregated
model. This model is then transmitted to all the clients in the local cluster.
The edge servers communicate to the central cloud server for global model
aggregation. The goal of each client is to converge to the global model, while
maintaining timeliness of the clients, i.e., having optimum training iteration
time. We investigate the convergence criteria for such a system with dense
clusters. Our analysis shows that for a system of $n$ clients with fixed
average timeliness, the convergence in finite time is probabilistically
guaranteed, if the nodes are divided into $O(1)$ number of clusters, that is,
if the system is built as a sparse set of edge servers with dense client bases
each.
Related papers
- Asynchronous Multi-Server Federated Learning for Geo-Distributed Clients [4.6792910030704515]
Federated learning (FL) systems enable multiple clients to train a machine learning model iteratively through synchronously exchanging the intermediate model weights with a single server.
The scalability of such FL systems can be limited by two factors: server idle time due to synchronous communication and the risk of a single server becoming the bottleneck.
We propose a new FL architecture that is entirely asynchronous, and therefore addresses these two limitations simultaneously.
arXiv Detail & Related papers (2024-06-03T15:29:46Z) - Towards Client Driven Federated Learning [7.528642177161784]
We introduce Client-Driven Federated Learning (CDFL), a novel FL framework that puts clients at the driving role.
In CDFL, each client independently and asynchronously updates its model by uploading the locally trained model to the server and receiving a customized model tailored to its local task.
arXiv Detail & Related papers (2024-05-24T10:17:49Z) - Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training.
In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework.
Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
arXiv Detail & Related papers (2023-08-11T09:58:47Z) - Optimizing Server-side Aggregation For Robust Federated Learning via
Subspace Training [80.03567604524268]
Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems.
We propose SmartFL, a generic approach that optimize the server-side aggregation process.
We provide theoretical analyses of the convergence and generalization capacity for SmartFL.
arXiv Detail & Related papers (2022-11-10T13:20:56Z) - Efficient Distribution Similarity Identification in Clustered Federated
Learning via Principal Angles Between Client Data Subspaces [59.33965805898736]
Clustered learning has been shown to produce promising results by grouping clients into clusters.
Existing FL algorithms are essentially trying to group clients together with similar distributions.
Prior FL algorithms attempt similarities indirectly during training.
arXiv Detail & Related papers (2022-09-21T17:37:54Z) - Client Selection Approach in Support of Clustered Federated Learning
over Wireless Edge Networks [2.6774008509840996]
Clustered Federated Multitask Learning (CFL) was introduced as an efficient scheme to obtain reliable specialized models.
This paper proposes a new client selection algorithm that aims to accelerate the convergence rate for obtaining specialized machine learning models.
arXiv Detail & Related papers (2021-08-16T21:38:22Z) - Hierarchical Quantized Federated Learning: Convergence Analysis and
System Design [7.481427303081613]
Federated learning is a collaborative machine to train deep neural networks without clients' private data.
Previous works assume one central parameter either at the cloud or at the edge.
This paper exploits the advantages of both cloud servers and considers Hierarchical Quantized Federated Learning system.
arXiv Detail & Related papers (2021-03-26T05:48:36Z) - A Bayesian Federated Learning Framework with Online Laplace
Approximation [144.7345013348257]
Federated learning allows multiple clients to collaboratively learn a globally shared model.
We propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side.
We achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.
arXiv Detail & Related papers (2021-02-03T08:36:58Z) - Timely Communication in Federated Learning [65.1253801733098]
We consider a global learning framework in which a parameter server (PS) trains a global model by using $n$ clients without actually storing the client data centrally at a cloud server.
Under the proposed scheme, at each iteration, the PS waits for $m$ available clients and sends them the current model.
We find the average age of information experienced by each client and numerically characterize the age-optimal $m$ and $k$ values for a given $n$.
arXiv Detail & Related papers (2020-12-31T18:52:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.