Federated Learning with Neural Graphical Models
- URL: http://arxiv.org/abs/2309.11680v2
- Date: Fri, 19 Jan 2024 19:53:21 GMT
- Title: Federated Learning with Neural Graphical Models
- Authors: Urszula Chajewska, Harsh Shrivastava
- Abstract summary: Federated Learning (FL) addresses the need to create models based on proprietary data.
Recent proposed Neural Graphical Models (NGMs) are Probabilistic Graphical models that utilize the expressive power of neural networks.
We develop a FL framework which maintains a global NGM model that learns the averaged information from the local NGM models.
- Score: 2.6842860806280058
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Federated Learning (FL) addresses the need to create models based on
proprietary data in such a way that multiple clients retain exclusive control
over their data, while all benefit from improved model accuracy due to pooled
resources. Recently proposed Neural Graphical Models (NGMs) are Probabilistic
Graphical models that utilize the expressive power of neural networks to learn
complex non-linear dependencies between the input features. They learn to
capture the underlying data distribution and have efficient algorithms for
inference and sampling. We develop a FL framework which maintains a global NGM
model that learns the averaged information from the local NGM models while
keeping the training data within the client's environment. Our design, FedNGMs,
avoids the pitfalls and shortcomings of neuron matching frameworks like
Federated Matched Averaging that suffers from model parameter explosion. Our
global model size remains constant throughout the process. In the cases where
clients have local variables that are not part of the combined global
distribution, we propose a `Stitching' algorithm, which personalizes the global
NGM models by merging the additional variables using the client's data. FedNGM
is robust to data heterogeneity, large number of participants, and limited
communication bandwidth.
Related papers
- An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets.
Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round.
We introduce FedAF, a novel aggregation-free FL algorithm.
arXiv Detail & Related papers (2024-04-29T05:55:23Z) - Federated Learning with Projected Trajectory Regularization [65.6266768678291]
Federated learning enables joint training of machine learning models from distributed clients without sharing their local data.
One key challenge in federated learning is to handle non-identically distributed data across the clients.
We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
arXiv Detail & Related papers (2023-12-22T02:12:08Z) - ProvFL: Client-Driven Interpretability of Global Model Predictions in
Federated Learning [0.0]
Federated Learning (FL) trains a collaborative machine learning model by aggregating multiple privately trained clients' models over several training rounds.
ProvFL is a fine-grained lineage capturing mechanism that tracks the flow of information between the individual participating clients in FL and the final global model.
ProvFL outperforms the state-of-the-art FL fault localization approach by an average margin of 50%.
arXiv Detail & Related papers (2023-12-21T07:48:54Z) - EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient
Federated Learning [15.124439914522693]
Federated Learning (FL) is a decentralized machine learning paradigm that enables collaborative model training across dispersed nodes.
This paper presents EvoFed, a novel approach that integrates Evolutionary Strategies (ES) with FL to address these challenges.
arXiv Detail & Related papers (2023-11-13T17:25:06Z) - Tunable Soft Prompts are Messengers in Federated Learning [55.924749085481544]
Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources.
The lack of model privacy protection in FL becomes an unneglectable challenge.
We propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts.
arXiv Detail & Related papers (2023-11-12T11:01:10Z) - Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training.
In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework.
Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
arXiv Detail & Related papers (2023-08-11T09:58:47Z) - FedDM: Iterative Distribution Matching for Communication-Efficient
Federated Learning [87.08902493524556]
Federated learning(FL) has recently attracted increasing attention from academia and industry.
We propose FedDM to build the global training objective from multiple local surrogate functions.
In detail, we construct synthetic sets of data on each client to locally match the loss landscape from original data.
arXiv Detail & Related papers (2022-07-20T04:55:18Z) - Fine-tuning Global Model via Data-Free Knowledge Distillation for
Non-IID Federated Learning [86.59588262014456]
Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint.
We propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG)
Our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
arXiv Detail & Related papers (2022-03-17T11:18:17Z) - Gradient Masked Averaging for Federated Learning [24.687254139644736]
Federated learning allows a large number of clients with heterogeneous data to coordinate learning of a unified global model.
Standard FL algorithms involve averaging of model parameters or gradient updates to approximate the global model at the server.
We propose a gradient masked averaging approach for FL as an alternative to the standard averaging of client updates.
arXiv Detail & Related papers (2022-01-28T08:42:43Z) - A Personalized Federated Learning Algorithm: an Application in Anomaly
Detection [0.6700873164609007]
Federated Learning (FL) has recently emerged as a promising method to overcome data privacy and transmission issues.
In FL, datasets collected from different devices or sensors are used to train local models (clients) each of which shares its learning with a centralized model (server)
This paper proposes a novel Personalized FedAvg (PC-FedAvg) which aims to control weights communication and aggregation augmented with a tailored learning algorithm to personalize the resulting models at each client.
arXiv Detail & Related papers (2021-11-04T04:57:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.