PersA-FL: Personalized Asynchronous Federated Learning
- URL: http://arxiv.org/abs/2210.01176v2
- Date: Wed, 4 Oct 2023 14:41:02 GMT
- Title: PersA-FL: Personalized Asynchronous Federated Learning
- Authors: Mohammad Taha Toghani, Soomin Lee and C\'esar A. Uribe
- Abstract summary: We study the personalized federated learning problem under asynchronous updates.
In this problem, each client seeks to obtain a personalized model that simultaneously outperforms local and global models.
We consider two optimization-based frameworks for personalization.
- Score: 1.1838866556981258
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: We study the personalized federated learning problem under asynchronous
updates. In this problem, each client seeks to obtain a personalized model that
simultaneously outperforms local and global models. We consider two
optimization-based frameworks for personalization: (i) Model-Agnostic
Meta-Learning (MAML) and (ii) Moreau Envelope (ME). MAML involves learning a
joint model adapted for each client through fine-tuning, whereas ME requires a
bi-level optimization problem with implicit gradients to enforce
personalization via regularized losses. We focus on improving the scalability
of personalized federated learning by removing the synchronous communication
assumption. Moreover, we extend the studied function class by removing
boundedness assumptions on the gradient norm. Our main technical contribution
is a unified proof for asynchronous federated learning with bounded staleness
that we apply to MAML and ME personalization frameworks. For the smooth and
non-convex functions class, we show the convergence of our method to a
first-order stationary point. We illustrate the performance of our method and
its tolerance to staleness through experiments for classification tasks over
heterogeneous datasets.
Related papers
- Personalized federated learning based on feature fusion [2.943623084019036]
Federated learning enables distributed clients to collaborate on training while storing their data locally to protect client privacy.
We propose a personalized federated learning approach called pFedPM.
In our process, we replace traditional gradient uploading with feature uploading, which helps reduce communication costs and allows for heterogeneous client models.
arXiv Detail & Related papers (2024-06-24T12:16:51Z) - MAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes [49.22075916259368]
In some real-world applications, data samples are usually distributed on local devices.
In this paper, we focus on a special kind of Non-I.I.D. scene where clients own incomplete classes.
Our proposed algorithm named MAP could simultaneously achieve the aggregation and personalization goals in FL.
arXiv Detail & Related papers (2024-04-14T12:22:42Z) - Learn What You Need in Personalized Federated Learning [53.83081622573734]
$textitLearn2pFed$ is a novel algorithm-unrolling-based personalized federated learning framework.
We show that $textitLearn2pFed$ significantly outperforms previous personalized federated learning methods.
arXiv Detail & Related papers (2024-01-16T12:45:15Z) - Elastically-Constrained Meta-Learner for Federated Learning [3.032797107899338]
Federated learning is an approach to collaboratively machine learning models for multiple parties that prohibit data sharing.
One of the challenges in federated learning is non-constrained data between clients, as a model can not fit data distribution for all clients.
arXiv Detail & Related papers (2023-06-29T05:58:47Z) - Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients.
FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification.
Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
arXiv Detail & Related papers (2023-05-01T20:04:46Z) - Model-Agnostic Multitask Fine-tuning for Few-shot Vision-Language
Transfer Learning [59.38343286807997]
We propose Model-Agnostic Multitask Fine-tuning (MAMF) for vision-language models on unseen tasks.
Compared with model-agnostic meta-learning (MAML), MAMF discards the bi-level optimization and uses only first-order gradients.
We show that MAMF consistently outperforms the classical fine-tuning method for few-shot transfer learning on five benchmark datasets.
arXiv Detail & Related papers (2022-03-09T17:26:53Z) - Adapt to Adaptation: Learning Personalization for Cross-Silo Federated
Learning [6.0088002781256185]
Conventional federated learning aims to train a global model for a federation of clients with decentralized data.
The distribution shift across non-IID datasets, also known as the data heterogeneity, often poses a challenge for this one-global-model-fits-all solution.
We propose APPLE, a personalized cross-silo FL framework that adaptively learns how much each client can benefit from other clients' models.
arXiv Detail & Related papers (2021-10-15T22:23:14Z) - QuPeL: Quantized Personalization with Applications to Federated Learning [8.420943739336067]
In this work, we introduce a textitquantized and textitpersonalized FL algorithm QuPeL that facilitates collective training with heterogeneous clients.
For personalization, we allow clients to learn textitcompressed personalized models with different quantization parameters depending on their resources.
Numerically, we show that optimizing over the quantization levels increases the performance and we validate that QuPeL outperforms both FedAvg and local training of clients in a heterogeneous setting.
arXiv Detail & Related papers (2021-02-23T16:43:51Z) - Toward Understanding the Influence of Individual Clients in Federated
Learning [52.07734799278535]
Federated learning allows clients to jointly train a global model without sending their private data to a central server.
We defined a new notion called em-Influence, quantify this influence over parameters, and proposed an effective efficient model to estimate this metric.
arXiv Detail & Related papers (2020-12-20T14:34:36Z) - Federated Mutual Learning [65.46254760557073]
Federated Mutual Leaning (FML) allows clients training a generalized model collaboratively and a personalized model independently.
The experiments show that FML can achieve better performance than alternatives in typical Federated learning setting.
arXiv Detail & Related papers (2020-06-27T09:35:03Z) - Personalized Federated Learning with Moreau Envelopes [16.25105865597947]
Federated learning (FL) is a decentralized and privacy-preserving machine learning technique.
One challenge associated with FL is statistical diversity among clients.
We propose an algorithm for personalized FL (FedFedMe) using envelopes regularized loss function.
arXiv Detail & Related papers (2020-06-16T00:55:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.