Clustered FedStack: Intermediate Global Models with Bayesian Information
Criterion
- URL: http://arxiv.org/abs/2309.11044v2
- Date: Sat, 14 Oct 2023 23:44:24 GMT
- Title: Clustered FedStack: Intermediate Global Models with Bayesian Information
Criterion
- Authors: Thanveer Shaik, Xiaohui Tao, Lin Li, Niall Higgins, Raj Gururajan,
Xujuan Zhou, Jianming Yong
- Abstract summary: We propose a novel Clustered FedStack framework based on the Stacked Federated Learning (FedStack) framework.
The local clients send their model predictions and output layer weights to a server, which then builds a robust global model.
This global model clusters the local clients based on their output layer weights using a clustering mechanism.
- Score: 8.478300563501035
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Federated Learning (FL) is currently one of the most popular technologies in
the field of Artificial Intelligence (AI) due to its collaborative learning and
ability to preserve client privacy. However, it faces challenges such as
non-identically and non-independently distributed (non-IID) and data with
imbalanced labels among local clients. To address these limitations, the
research community has explored various approaches such as using local model
parameters, federated generative adversarial learning, and federated
representation learning. In our study, we propose a novel Clustered FedStack
framework based on the previously published Stacked Federated Learning
(FedStack) framework. The local clients send their model predictions and output
layer weights to a server, which then builds a robust global model. This global
model clusters the local clients based on their output layer weights using a
clustering mechanism. We adopt three clustering mechanisms, namely K-Means,
Agglomerative, and Gaussian Mixture Models, into the framework and evaluate
their performance. We use Bayesian Information Criterion (BIC) with the maximum
likelihood function to determine the number of clusters. The Clustered FedStack
models outperform baseline models with clustering mechanisms. To estimate the
convergence of our proposed framework, we use Cyclical learning rates.
Related papers
- FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering [26.478852701376294]
Federated learning (FL) is an emerging distributed machine learning paradigm.
One of the major challenges in FL is the presence of uneven data distributions across client devices.
We propose em FedClust, a novel approach for CFL that leverages the correlation between local model weights and the data distribution of clients.
arXiv Detail & Related papers (2024-07-09T02:47:16Z) - An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets.
Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round.
We introduce FedAF, a novel aggregation-free FL algorithm.
arXiv Detail & Related papers (2024-04-29T05:55:23Z) - FedClust: Optimizing Federated Learning on Non-IID Data through
Weight-Driven Client Clustering [28.057411252785176]
Federated learning (FL) is an emerging distributed machine learning paradigm enabling collaborative model training on decentralized devices without exposing their local data.
This paper proposes FedClust, a novel CFL approach leveraging correlations between local model weights and client data distributions.
arXiv Detail & Related papers (2024-03-07T01:50:36Z) - Federated cINN Clustering for Accurate Clustered Federated Learning [33.72494731516968]
Federated Learning (FL) presents an innovative approach to privacy-preserving distributed machine learning.
We propose the Federated cINN Clustering Algorithm (FCCA) to robustly cluster clients into different groups.
arXiv Detail & Related papers (2023-09-04T10:47:52Z) - Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection.
We find that the difference in logits between the local and global models increases as the model is continuously updated.
We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
arXiv Detail & Related papers (2023-08-20T04:41:01Z) - Timely Asynchronous Hierarchical Federated Learning: Age of Convergence [59.96266198512243]
We consider an asynchronous hierarchical federated learning setting with a client-edge-cloud framework.
The clients exchange the trained parameters with their corresponding edge servers, which update the locally aggregated model.
The goal of each client is to converge to the global model, while maintaining timeliness of the clients.
arXiv Detail & Related papers (2023-06-21T17:39:16Z) - Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device)
In FL, each data holder trains a model locally and releases it to a central server for aggregation.
In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation).
In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
arXiv Detail & Related papers (2022-11-20T10:49:22Z) - Personalized Federated Learning with Multiple Known Clusters [20.585114235701603]
We consider the problem of personalized federated learning when there are known cluster structures within users.
An intuitive approach would be to regularize the parameters so that users in the same cluster share similar model weights.
We develop an algorithm that allows each cluster to communicate independently and derive the convergence results.
arXiv Detail & Related papers (2022-04-28T16:32:29Z) - Cluster-driven Graph Federated Learning over Multiple Domains [25.51716405561116]
Graph Federated Learning (FL) deals with learning a central model (i.e. the server) in privacy-constrained scenarios.
Here we propose a novel Cluster-driven Graph Federated Learning (FedCG)
arXiv Detail & Related papers (2021-04-29T19:31:19Z) - A Bayesian Federated Learning Framework with Online Laplace
Approximation [144.7345013348257]
Federated learning allows multiple clients to collaboratively learn a globally shared model.
We propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side.
We achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.
arXiv Detail & Related papers (2021-02-03T08:36:58Z) - Edge-assisted Democratized Learning Towards Federated Analytics [67.44078999945722]
We show the hierarchical learning structure of the proposed edge-assisted democratized learning mechanism, namely Edge-DemLearn.
We also validate Edge-DemLearn as a flexible model training mechanism to build a distributed control and aggregation methodology in regions.
arXiv Detail & Related papers (2020-12-01T11:46:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.