A Simple Data Augmentation for Feature Distribution Skewed Federated Learning

• URL: http://arxiv.org/abs/2306.09363v2
• Date: Fri, 06 Dec 2024 05:35:09 GMT
• Title: A Simple Data Augmentation for Feature Distribution Skewed Federated Learning
• Authors: Yunlu Yan, Huazhu Fu, Yuexiang Li, Jinheng Xie, Jun Ma, Guang Yang, Lei Zhu,
• Abstract summary: Federated Learning (FL) facilitates collaborative learning among multiple clients in a distributed manner. FL's performance degrades with non-Independent and Identically Distributed (non-IID) data. We propose FedRDN, which randomly injects the statistical information of the local distribution from the entire federation into the client's data. Our FedRDN is a plug-and-play component, which can be seamlessly integrated into the data augmentation flow with only a few lines of code.
• Score: 47.27053883247425
• License:
• Abstract: Federated Learning (FL) facilitates collaborative learning among multiple clients in a distributed manner and ensures the security of privacy. However, its performance inevitably degrades with non-Independent and Identically Distributed (non-IID) data. In this paper, we focus on the feature distribution skewed FL scenario, a common non-IID situation in real-world applications where data from different clients exhibit varying underlying distributions. This variation leads to feature shift, which is a key issue of this scenario. While previous works have made notable progress, few pay attention to the data itself, i.e., the root of this issue. The primary goal of this paper is to mitigate feature shift from the perspective of data. To this end, we propose a simple yet remarkably effective input-level data augmentation method, namely FedRDN, which randomly injects the statistical information of the local distribution from the entire federation into the client's data. This is beneficial to improve the generalization of local feature representations, thereby mitigating feature shift. Moreover, our FedRDN is a plug-and-play component, which can be seamlessly integrated into the data augmentation flow with only a few lines of code. Extensive experiments on several datasets show that the performance of various representative FL methods can be further improved by integrating our FedRDN, demonstrating its effectiveness, strong compatibility and generalizability. Code will be released.

Related papers

• Disentangling data distribution for Federated Learning [20.524108508314107]
  Federated Learning (FL) facilitates collaborative training of a global model whose performance is boosted by private data owned by distributed clients. Yet the wide applicability of FL is hindered by entanglement of data distributions across different clients. This paper demonstrates for the first time that by disentangling data distributions FL can in principle achieve efficiencies comparable to those of distributed systems.
  arXiv  Detail & Related papers  (2024-10-16T13:10:04Z)
• StatAvg: Mitigating Data Heterogeneity in Federated Learning for Intrusion Detection Systems [22.259297167311964]
  Federated learning (FL) is a decentralized learning technique that enables devices to collaboratively build a shared Machine Leaning (ML) or Deep Learning (DL) model without revealing their raw data to a third party. Due to its privacy-preserving nature, FL has sparked widespread attention for building Intrusion Detection Systems (IDS) within the realm of cybersecurity. We propose an effective method called Statistical Averaging (StatAvg) to alleviate non-independently and identically (non-iid) distributed features across local clients' data in FL.
  arXiv  Detail & Related papers  (2024-05-20T14:41:59Z)
• FLASH: Federated Learning Across Simultaneous Heterogeneities [54.80435317208111]
  FLASH(Federated Learning Across Simultaneous Heterogeneities) is a lightweight and flexible client selection algorithm. It outperforms state-of-the-art FL frameworks under extensive sources of Heterogeneities. It achieves substantial and consistent improvements over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-13T20:04:39Z)
• FedFed: Feature Distillation against Data Heterogeneity in Federated Learning [88.36513907827552]
  Federated learning (FL) typically faces data heterogeneity, i.e., distribution shifting among clients. We propose a novel approach called textbfFederated textbfFeature textbfdistillation (FedFedFed) FedFed partitions data into performance-sensitive features (i.e., greatly contributing to model performance) and performance-robust features (i.e., limitedly contributing to model performance) Comprehensive experiments demonstrate the efficacy of FedFed in promoting model performance.
  arXiv  Detail & Related papers  (2023-10-08T09:00:59Z)
• PS-FedGAN: An Efficient Federated Learning Framework Based on Partially Shared Generative Adversarial Networks For Data Privacy [56.347786940414935]
  Federated Learning (FL) has emerged as an effective learning paradigm for distributed computation. This work proposes a novel FL framework that requires only partial GAN model sharing. Named as PS-FedGAN, this new framework enhances the GAN releasing and training mechanism to address heterogeneous data distributions.
  arXiv  Detail & Related papers  (2023-05-19T05:39:40Z)
• Benchmarking FedAvg and FedCurv for Image Classification Tasks [1.376408511310322]
  This paper focuses on the problem of statistical heterogeneity of the data in the same federated network. Several Federated Learning algorithms, such as FedAvg, FedProx and Federated Curvature (FedCurv) have already been proposed. As a side product of this work, we release the non-IID version of the datasets we used so to facilitate further comparisons from the FL community.
  arXiv  Detail & Related papers  (2023-03-31T10:13:01Z)
• FedFA: Federated Feature Augmentation [25.130087374092383]
  Federated learning allows multiple parties to collaboratively train deep models without exchanging raw data. The primary goal of this paper is to develop a robust federated learning algorithm to address feature shift in clients' samples. We propose FedFA to tackle federated learning from a distinct perspective of federated feature augmentation.
  arXiv  Detail & Related papers  (2023-01-30T15:39:55Z)
• FedAvg with Fine Tuning: Local Updates Lead to Representation Learning [54.65133770989836]
  Federated Averaging (FedAvg) algorithm consists of alternating between a few local gradient updates at client nodes, followed by a model averaging update at the server. We show that the reason behind generalizability of the FedAvg's output is its power in learning the common data representation among the clients' tasks. We also provide empirical evidence demonstrating FedAvg's representation learning ability in federated image classification with heterogeneous data.
  arXiv  Detail & Related papers  (2022-05-27T00:55:24Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• FedBN: Federated Learning on Non-IID Features via Local Batch Normalization [23.519212374186232]
  The emerging paradigm of federated learning (FL) strives to enable collaborative training of deep models on the network edge without centrally aggregating raw data. We propose an effective method that uses local batch normalization to alleviate the feature shift before averaging models. The resulting scheme, called FedBN, outperforms both classical FedAvg and the state-of-the-art for non-iid data.
  arXiv  Detail & Related papers  (2021-02-15T16:04:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.