FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation

• URL: http://arxiv.org/abs/2312.02327v2
• Date: Tue, 2 Jul 2024 01:37:34 GMT
• Title: FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation
• Authors: Tong Xia, Abhirup Ghosh, Xinchi Qiu, Cecilia Mascolo,
• Abstract summary: Federated Learning (FL) enables model development by leveraging data distributed across numerous edge devices without transferring local data to a central server. Existing FL methods face challenges when dealing with scarce and label-skewed data across devices, resulting in local model overfitting and drift. We propose a pioneering framework called textitFLea, incorporating the following key components.
• Score: 15.298650496155508
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Federated Learning (FL) enables model development by leveraging data distributed across numerous edge devices without transferring local data to a central server. However, existing FL methods still face challenges when dealing with scarce and label-skewed data across devices, resulting in local model overfitting and drift, consequently hindering the performance of the global model. In response to these challenges, we propose a pioneering framework called \textit{FLea}, incorporating the following key components: \textit{i)} A global feature buffer that stores activation-target pairs shared from multiple clients to support local training. This design mitigates local model drift caused by the absence of certain classes; \textit{ii)} A feature augmentation approach based on local and global activation mix-ups for local training. This strategy enlarges the training samples, thereby reducing the risk of local overfitting; \textit{iii)} An obfuscation method to minimize the correlation between intermediate activations and the source data, enhancing the privacy of shared features. To verify the superiority of \textit{FLea}, we conduct extensive experiments using a wide range of data modalities, simulating different levels of local data scarcity and label skew. The results demonstrate that \textit{FLea} consistently outperforms state-of-the-art FL counterparts (among 13 of the experimented 18 settings, the improvement is over $5\%$) while concurrently mitigating the privacy vulnerabilities associated with shared features. Code is available at https://github.com/XTxiatong/FLea.git

Related papers

• FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation [15.298650496155508]
  Federated Learning (FL) enables model development by leveraging data distributed across numerous edge devices without transferring local data to a central server. Existing FL methods face challenges when dealing with scarce and label-skewed data across devices, resulting in local model overfitting and drift. We propose a pioneering framework called FLea, incorporating the following key components.
  arXiv  Detail & Related papers  (2024-06-13T19:28:08Z)
• Tunable Soft Prompts are Messengers in Federated Learning [55.924749085481544]
  Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources. The lack of model privacy protection in FL becomes an unneglectable challenge. We propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts.
  arXiv  Detail & Related papers  (2023-11-12T11:01:10Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reduction [5.757705591791482]
  Federated Learning (FL) is a way for machines to learn from data that is kept locally, in order to protect the privacy of clients. We propose FedBR, a novel unified algorithm that reduces the local learning bias on features and classifiers. We conducted several experiments to test algopt and found that it consistently outperforms other SOTA FL methods.
  arXiv  Detail & Related papers  (2022-05-26T16:08:01Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning [61.488646649045215]
  Federated learning (FL) is a promising strategy for performing privacy-preserving, distributed learning with a network of clients (i.e., edge devices)
  arXiv  Detail & Related papers  (2021-11-28T19:03:39Z)
• Multi-Center Federated Learning [62.32725938999433]
  Federated learning (FL) can protect data privacy in distributed learning. It merely collects local gradients from users without access to their data. We propose a novel multi-center aggregation mechanism.
  arXiv  Detail & Related papers  (2021-08-19T12:20:31Z)
• FedMix: Approximation of Mixup under Mean Augmented Federated Learning [60.503258658382]
  Federated learning (FL) allows edge devices to collectively learn a model without directly sharing data within each device. Current state-of-the-art algorithms suffer from performance degradation as the heterogeneity of local data across clients increases. We propose a new augmentation algorithm, named FedMix, which is inspired by a phenomenal yet simple data augmentation method, Mixup.
  arXiv  Detail & Related papers  (2021-07-01T06:14:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.