PGFed: Personalize Each Client's Global Objective for Federated Learning

• URL: http://arxiv.org/abs/2212.01448v2
• Date: Thu, 7 Sep 2023 16:01:15 GMT
• Title: PGFed: Personalize Each Client's Global Objective for Federated Learning
• Authors: Jun Luo, Matias Mendieta, Chen Chen, Shandong Wu
• Abstract summary: We propose a novel personalized FL framework that enables each client to personalize its own global objective. To avoid massive (O(N2)) communication overhead and potential privacy leakage, each client's risk is estimated through a first-order approximation for other clients' adaptive risk aggregation. Our experiments on four datasets under different federated settings show consistent improvements of PGFed over previous state-of-the-art methods.
• Score: 7.810284483002312
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Personalized federated learning has received an upsurge of attention due to the mediocre performance of conventional federated learning (FL) over heterogeneous data. Unlike conventional FL which trains a single global consensus model, personalized FL allows different models for different clients. However, existing personalized FL algorithms only implicitly transfer the collaborative knowledge across the federation by embedding the knowledge into the aggregated model or regularization. We observed that this implicit knowledge transfer fails to maximize the potential of each client's empirical risk toward other clients. Based on our observation, in this work, we propose Personalized Global Federated Learning (PGFed), a novel personalized FL framework that enables each client to personalize its own global objective by explicitly and adaptively aggregating the empirical risks of itself and other clients. To avoid massive (O(N^2)) communication overhead and potential privacy leakage while achieving this, each client's risk is estimated through a first-order approximation for other clients' adaptive risk aggregation. On top of PGFed, we develop a momentum upgrade, dubbed PGFedMo, to more efficiently utilize clients' empirical risks. Our extensive experiments on four datasets under different federated settings show consistent improvements of PGFed over previous state-of-the-art methods. The code is publicly available at https://github.com/ljaiverson/pgfed.

Related papers

• Personalized Federated Knowledge Graph Embedding with Client-Wise Relation Graph [49.66272783945571]
  We propose Personalized Federated knowledge graph Embedding with client-wise relation graph. PFedEG learns personalized supplementary knowledge for each client by amalgamating entity embedding from its neighboring clients. We conduct extensive experiments on four benchmark datasets to evaluate our method against state-of-the-art models.
  arXiv  Detail & Related papers  (2024-06-17T17:44:53Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• PFL-GAN: When Client Heterogeneity Meets Generative Models in Personalized Federated Learning [55.930403371398114]
  We propose a novel generative adversarial network (GAN) sharing and aggregation strategy for personalized learning (PFL) PFL-GAN addresses the client heterogeneity in different scenarios. More specially, we first learn the similarity among clients and then develop an weighted collaborative data aggregation. The empirical results through the rigorous experimentation on several well-known datasets demonstrate the effectiveness of PFL-GAN.
  arXiv  Detail & Related papers  (2023-08-23T22:38:35Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• Personalized Federated Learning via Amortized Bayesian Meta-Learning [21.126405589760367]
  We introduce a new perspective on personalized federated learning through Amortized Bayesian Meta-Learning. Specifically, we propose a novel algorithm called emphFedABML, which employs hierarchical variational inference across clients. Our theoretical analysis provides an upper bound on the average generalization error and guarantees the generalization performance on unseen data.
  arXiv  Detail & Related papers  (2023-07-05T11:58:58Z)
• FedJETs: Efficient Just-In-Time Personalization with Federated Mixture of Experts [48.78037006856208]
  FedJETs is a novel solution by using a Mixture-of-Experts (MoE) framework within a Federated Learning (FL) setup. Our method leverages the diversity of the clients to train specialized experts on different subsets of classes, and a gating function to route the input to the most relevant expert(s) Our approach can improve accuracy up to 18% in state of the art FL settings, while maintaining competitive zero-shot performance.
  arXiv  Detail & Related papers  (2023-06-14T15:47:52Z)
• Confidence-aware Personalized Federated Learning via Variational Expectation Maximization [34.354154518009956]
  We present a novel framework for personalized Federated Learning (PFL) PFL is a distributed learning scheme to train a shared model across clients. We present a novel framework for PFL based on hierarchical modeling and variational inference.
  arXiv  Detail & Related papers  (2023-05-21T20:12:27Z)
• Personalized Federated Learning on Long-Tailed Data via Adversarial Feature Augmentation [24.679535905451758]
  PFL aims to learn personalized models for each client based on the knowledge across all clients in a privacy-preserving manner. Existing PFL methods assume that the underlying global data across all clients are uniformly distributed without considering the long-tail distribution. We propose Federated Learning with Adversarial Feature Augmentation (FedAFA) to address this joint problem in PFL.
  arXiv  Detail & Related papers  (2023-03-27T13:00:20Z)
• Personalized Privacy-Preserving Framework for Cross-Silo Federated Learning [0.0]
  Federated learning (FL) is a promising decentralized deep learning (DL) framework that enables DL-based approaches trained collaboratively across clients without sharing private data. In this paper, we propose a novel framework, namely Personalized Privacy-Preserving Federated Learning (PPPFL) Our proposed framework outperforms multiple FL baselines on different datasets, including MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100.
  arXiv  Detail & Related papers  (2023-02-22T07:24:08Z)
• To Federate or Not To Federate: Incentivizing Client Participation in Federated Learning [22.3101738137465]
  Federated learning (FL) facilitates collaboration between a group of clients who seek to train a common machine learning model. In this paper, we propose an algorithm called IncFL that explicitly maximizes the fraction of clients who are incentivized to use the global model.
  arXiv  Detail & Related papers  (2022-05-30T04:03:31Z)
• Personalized Federated Learning with First Order Model Optimization [76.81546598985159]
  We propose an alternative to federated learning, where each client federates with other relevant clients to obtain a stronger model per client-specific objectives. We do not assume knowledge of underlying data distributions or client similarities, and allow each client to optimize for arbitrary target distributions of interest. Our method outperforms existing alternatives, while also enabling new features for personalized FL such as transfer outside of local data distributions.
  arXiv  Detail & Related papers  (2020-12-15T19:30:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.