Related papers: Personalized Federated Learning via Amortized Bayesian Meta-Learning

Personalized Federated Learning via Amortized Bayesian Meta-Learning

URL: http://arxiv.org/abs/2307.02222v1
Date: Wed, 5 Jul 2023 11:58:58 GMT
Title: Personalized Federated Learning via Amortized Bayesian Meta-Learning
Authors: Shiyu Liu, Shaogao Lv, Dun Zeng, Zenglin Xu, Hui Wang and Yue Yu
Abstract summary: We introduce a new perspective on personalized federated learning through Amortized Bayesian Meta-Learning. Specifically, we propose a novel algorithm called emphFedABML, which employs hierarchical variational inference across clients. Our theoretical analysis provides an upper bound on the average generalization error and guarantees the generalization performance on unseen data.
Score: 21.126405589760367
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Federated learning is a decentralized and privacy-preserving technique that enables multiple clients to collaborate with a server to learn a global model without exposing their private data. However, the presence of statistical heterogeneity among clients poses a challenge, as the global model may struggle to perform well on each client's specific task. To address this issue, we introduce a new perspective on personalized federated learning through Amortized Bayesian Meta-Learning. Specifically, we propose a novel algorithm called \emph{FedABML}, which employs hierarchical variational inference across clients. The global prior aims to capture representations of common intrinsic structures from heterogeneous clients, which can then be transferred to their respective tasks and aid in the generation of accurate client-specific approximate posteriors through a few local updates. Our theoretical analysis provides an upper bound on the average generalization error and guarantees the generalization performance on unseen data. Finally, several empirical results are implemented to demonstrate that \emph{FedABML} outperforms several competitive baselines.

Related papers

Personalized Federated Knowledge Graph Embedding with Client-Wise Relation Graph [49.66272783945571]
We propose Personalized Federated knowledge graph Embedding with client-wise relation graph. PFedEG learns personalized supplementary knowledge for each client by amalgamating entity embedding from its neighboring clients. We conduct extensive experiments on four benchmark datasets to evaluate our method against state-of-the-art models.
arXiv Detail & Related papers (2024-06-17T17:44:53Z)
Personalized Federated Learning via Sequential Layer Expansion in Representation Learning [0.0]
Federated learning ensures the privacy of clients by conducting distributed training on individual client devices and sharing only the model weights with a central server. We propose a new representation learning-based approach that suggests decoupling the entire deep learning model into more densely divided parts with the application of suitable scheduling methods.
arXiv Detail & Related papers (2024-04-27T06:37:19Z)
FedSampling: A Better Sampling Strategy for Federated Learning [81.85411484302952]
Federated learning (FL) is an important technique for learning models from decentralized data in a privacy-preserving way. Existing FL methods usually uniformly sample clients for local model learning in each round. We propose a novel data uniform sampling strategy for federated learning (FedSampling)
arXiv Detail & Related papers (2023-06-25T13:38:51Z)
Personalized Federated Learning with Feature Alignment and Classifier Collaboration [13.320381377599245]
Data heterogeneity is one of the most challenging issues in federated learning. One such approach in deep neural networks based tasks is employing a shared feature representation and learning a customized classifier head for each client. In this work, we conduct explicit local-global feature alignment by leveraging global semantic knowledge for learning a better representation.
arXiv Detail & Related papers (2023-06-20T19:58:58Z)
Straggler-Resilient Personalized Federated Learning [55.54344312542944]
Federated learning allows training models from samples distributed across a large network of clients while respecting privacy and communication restrictions. We develop a novel algorithmic procedure with theoretical speedup guarantees that simultaneously handles two of these hurdles. Our method relies on ideas from representation learning theory to find a global common representation using all clients' data and learn a user-specific set of parameters leading to a personalized solution for each client.
arXiv Detail & Related papers (2022-06-05T01:14:46Z)
DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
arXiv Detail & Related papers (2022-04-16T08:08:29Z)
Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity [3.291862617649511]
We propose a new approach for obtaining a personalized model from a client-level objective. To realize this personalization, we leverage finding a small subnetwork for each client.
arXiv Detail & Related papers (2021-05-02T22:10:46Z)
Exploiting Shared Representations for Personalized Federated Learning [54.65133770989836]
We propose a novel federated learning framework and algorithm for learning a shared data representation across clients and unique local heads for each client. Our algorithm harnesses the distributed computational power across clients to perform many local-updates with respect to the low-dimensional local parameters for every update of the representation. This result is of interest beyond federated learning to a broad class of problems in which we aim to learn a shared low-dimensional representation among data distributions.
arXiv Detail & Related papers (2021-02-14T05:36:25Z)
Toward Understanding the Influence of Individual Clients in Federated Learning [52.07734799278535]
Federated learning allows clients to jointly train a global model without sending their private data to a central server. We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
arXiv Detail & Related papers (2020-12-20T14:34:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.