Recovering Global Data Distribution Locally in Federated Learning

• URL: http://arxiv.org/abs/2409.14063v1
• Date: Sat, 21 Sep 2024 08:35:04 GMT
• Title: Recovering Global Data Distribution Locally in Federated Learning
• Authors: Ziyu Yao,
• Abstract summary: Federated Learning (FL) is a distributed machine learning paradigm that enables collaboration among multiple clients. A major challenge in FL is the label imbalance, where clients may exclusively possess certain classes while having numerous minority and missing classes. We propose a novel approach ReGL to address this challenge, whose key idea is to Recover the Global data distribution Locally.
• Score: 7.885010255812708
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated Learning (FL) is a distributed machine learning paradigm that enables collaboration among multiple clients to train a shared model without sharing raw data. However, a major challenge in FL is the label imbalance, where clients may exclusively possess certain classes while having numerous minority and missing classes. Previous works focus on optimizing local updates or global aggregation but ignore the underlying imbalanced label distribution across clients. In this paper, we propose a novel approach ReGL to address this challenge, whose key idea is to Recover the Global data distribution Locally. Specifically, each client uses generative models to synthesize images that complement the minority and missing classes, thereby alleviating label imbalance. Moreover, we adaptively fine-tune the image generation process using local real data, which makes the synthetic images align more closely with the global distribution. Importantly, both the generation and fine-tuning processes are conducted at the client-side without leaking data privacy. Through comprehensive experiments on various image classification datasets, we demonstrate the remarkable superiority of our approach over existing state-of-the-art works in fundamentally tackling label imbalance in FL.

Related papers

• Personalized Federated Learning via Feature Distribution Adaptation [3.410799378893257]
  Federated learning (FL) is a distributed learning framework that leverages commonalities between distributed client datasets to train a global model. personalized federated learning (PFL) seeks to address this by learning individual models tailored to each client. We propose an algorithm, pFedFDA, that efficiently generates personalized models by adapting global generative classifiers to their local feature distributions.
  arXiv  Detail & Related papers  (2024-11-01T03:03:52Z)
• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity [50.44021981013037]
  Federated Learning (FL) relies on the effectiveness of utilizing knowledge from distributed datasets. Traditional FL methods adopt an aggregate-then-adapt framework, where clients update local models based on a global model aggregated by the server from the previous training round. We introduce FedAF, a novel aggregation-free FL algorithm.
  arXiv  Detail & Related papers  (2024-04-29T05:55:23Z)
• Exploring Vacant Classes in Label-Skewed Federated Learning [113.65301899666645]
  Label skews, characterized by disparities in local label distribution across clients, pose a significant challenge in federated learning. This paper introduces FedVLS, a novel approach to label-skewed federated learning that integrates vacant-class distillation and logit suppression simultaneously.
  arXiv  Detail & Related papers  (2024-01-04T16:06:31Z)
• Federated Skewed Label Learning with Logits Fusion [23.062650578266837]
  Federated learning (FL) aims to collaboratively train a shared model across multiple clients without transmitting their local data. We propose FedBalance, which corrects the optimization bias among local models by calibrating their logits. Our method can gain 13% higher average accuracy compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-11-14T14:37:33Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• Towards Unbiased Training in Federated Open-world Semi-supervised Learning [15.08153616709326]
  We propose a novel Federatedopen-world Semi-Supervised Learning (FedoSSL) framework, which can solve the key challenge in distributed and open-world settings. We adopt an uncertainty-aware suppressed loss to alleviate the biased training between locally unseen and globally unseen classes. The proposed FedoSSL can be easily adapted to state-of-the-art FL methods, which is also validated via extensive experiments on benchmarks and real-world datasets.
  arXiv  Detail & Related papers  (2023-05-01T11:12:37Z)
• No One Left Behind: Real-World Federated Class-Incremental Learning [111.77681016996202]
  Local-Global Anti-forgetting (LGA) model addresses local and global catastrophic forgetting. We develop a category-balanced gradient-adaptive compensation loss and a category gradient-induced semantic distillation loss. It augments perturbed prototype images of new categories collected from local clients via self-supervised prototype augmentation.
  arXiv  Detail & Related papers  (2023-02-02T06:41:02Z)
• Integrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning [60.41501515192088]
  Federated Learning (FL) has become a popular distributed learning paradigm that involves multiple clients training a global model collaboratively. The data samples usually follow a long-tailed distribution in the real world, and FL on the decentralized and long-tailed data yields a poorly-behaved global model. In this work, we integrate the local real data with the global gradient prototypes to form the local balanced datasets.
  arXiv  Detail & Related papers  (2023-01-25T03:18:10Z)
• Modeling Global Distribution for Federated Learning with Label Distribution Skew [15.417187554408104]
  Federated learning achieves joint training of deep models by connecting decentralized data sources. In a more general case, the distributions of labels among clients are different, called label distribution skew'' We propose a novel federated learning method, named FedMGD, to alleviate the performance degradation caused by the label distribution skew issue.
  arXiv  Detail & Related papers  (2022-12-17T14:46:01Z)
• DYNAFED: Tackling Client Data Heterogeneity with Global Dynamics [60.60173139258481]
  Local training on non-iid distributed data results in deflected local optimum. A natural solution is to gather all client data onto the server, such that the server has a global view of the entire data distribution. In this paper, we put forth an idea to collect and leverage global knowledge on the server without hindering data privacy.
  arXiv  Detail & Related papers  (2022-11-20T06:13:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.