Advocating for the Silent: Enhancing Federated Generalization for Non-Participating Clients

• URL: http://arxiv.org/abs/2310.07171v6
• Date: Thu, 10 Oct 2024 13:58:08 GMT
• Title: Advocating for the Silent: Enhancing Federated Generalization for Non-Participating Clients
• Authors: Zheshun Wu, Zenglin Xu, Dun Zeng, Qifan Wang, Jie Liu,
• Abstract summary: This paper unveils an information-theoretic generalization framework for Federated Learning. It quantifies generalization errors by evaluating the information entropy of local distributions. Inspired by our deduced generalization bounds, we introduce a weighted aggregation approach and a duo of client selection strategies.
• Score: 38.804196122833645
• License:
• Abstract: Federated Learning (FL) has surged in prominence due to its capability of collaborative model training without direct data sharing. However, the vast disparity in local data distributions among clients, often termed the Non-Independent Identically Distributed (Non-IID) challenge, poses a significant hurdle to FL's generalization efficacy. The scenario becomes even more complex when not all clients participate in the training process, a common occurrence due to unstable network connections or limited computational capacities. This can greatly complicate the assessment of the trained models' generalization abilities. While a plethora of recent studies has centered on the generalization gap pertaining to unseen data from participating clients with diverse distributions, the distinction between the training distributions of participating clients and the testing distributions of non-participating ones has been largely overlooked. In response, our paper unveils an information-theoretic generalization framework for FL. Specifically, it quantifies generalization errors by evaluating the information entropy of local distributions and discerning discrepancies across these distributions. Inspired by our deduced generalization bounds, we introduce a weighted aggregation approach and a duo of client selection strategies. These innovations are designed to strengthen FL's ability to generalize and thus ensure that trained models perform better on non-participating clients by incorporating a more diverse range of client data distributions. Our extensive empirical evaluations reaffirm the potency of our proposed methods, aligning seamlessly with our theoretical construct.

Related papers

• Personalized Federated Learning via Feature Distribution Adaptation [3.410799378893257]
  Federated learning (FL) is a distributed learning framework that leverages commonalities between distributed client datasets to train a global model. personalized federated learning (PFL) seeks to address this by learning individual models tailored to each client. We propose an algorithm, pFedFDA, that efficiently generates personalized models by adapting global generative classifiers to their local feature distributions.
  arXiv  Detail & Related papers  (2024-11-01T03:03:52Z)
• Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchors [21.931436901703634]
  Conventional Federated Learning (FL) involves collaborative training of a global model while maintaining user data privacy. One of its branches, decentralized FL, is a serverless network that allows clients to own and optimize different local models separately. We propose a novel Decentralized FL technique by introducing Synthetic Anchors, dubbed as DeSA.
  arXiv  Detail & Related papers  (2024-05-19T11:36:45Z)
• FedImpro: Measuring and Improving Client Update in Federated Learning [77.68805026788836]
  Federated Learning (FL) models often experience client drift caused by heterogeneous data. We present an alternative perspective on client drift and aim to mitigate it by generating improved local models.
  arXiv  Detail & Related papers  (2024-02-10T18:14:57Z)
• FedCiR: Client-Invariant Representation Learning for Federated Non-IID Features [15.555538379806135]
  Federated learning (FL) is a distributed learning paradigm that maximizes the potential of data-driven models for edge devices without sharing their raw data. We propose FedCiR, a client-invariant representation learning framework that enables clients to extract informative and client-invariant features.
  arXiv  Detail & Related papers  (2023-08-30T06:36:32Z)
• FedABC: Targeting Fair Competition in Personalized Federated Learning [76.9646903596757]
  Federated learning aims to collaboratively train models without accessing their client's local private data. We propose a novel and generic PFL framework termed Federated Averaging via Binary Classification, dubbed FedABC. In particular, we adopt the one-vs-all'' training strategy in each client to alleviate the unfair competition between classes.
  arXiv  Detail & Related papers  (2023-02-15T03:42:59Z)
• Integrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning [60.41501515192088]
  Federated Learning (FL) has become a popular distributed learning paradigm that involves multiple clients training a global model collaboratively. The data samples usually follow a long-tailed distribution in the real world, and FL on the decentralized and long-tailed data yields a poorly-behaved global model. In this work, we integrate the local real data with the global gradient prototypes to form the local balanced datasets.
  arXiv  Detail & Related papers  (2023-01-25T03:18:10Z)
• Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learning [13.246981646250518]
  This paper presents a general dual-regularized learning framework to explore the personalized invariance, compared with the exsiting personalized federated learning methods. We show that our method is superior over the existing federated learning and invariant learning methods, in diverse out-of-distribution and Non-IID data cases.
  arXiv  Detail & Related papers  (2022-11-21T08:17:03Z)
• FedGen: Generalizable Federated Learning for Sequential Data [8.784435748969806]
  In many real-world distributed settings, spurious correlations exist due to biases and data sampling issues. We present a generalizable federated learning framework called FedGen, which allows clients to identify and distinguish between spurious and invariant features. We show that FedGen results in models that achieve significantly better generalization and can outperform the accuracy of current federated learning approaches by over 24%.
  arXiv  Detail & Related papers  (2022-11-03T15:48:14Z)
• FL Games: A Federated Learning Framework for Distribution Shifts [71.98708418753786]
  Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server. We propose FL GAMES, a game-theoretic framework for federated learning that learns causal features that are invariant across clients.
  arXiv  Detail & Related papers  (2022-10-31T22:59:03Z)
• FL Games: A federated learning framework for distribution shifts [71.98708418753786]
  Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server. We propose FL Games, a game-theoretic framework for federated learning for learning causal features that are invariant across clients.
  arXiv  Detail & Related papers  (2022-05-23T07:51:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.