Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation

• URL: http://arxiv.org/abs/2407.16139v1
• Date: Tue, 23 Jul 2024 02:52:52 GMT
• Title: Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation
• Authors: Xinghao Wu, Jianwei Niu, Xuefeng Liu, Mingjia Shi, Guogang Zhu, Shaojie Tang,
• Abstract summary: In traditional Federated Learning approaches, the global model underperforms when faced with data heterogeneity. We propose a new PFL framework called FedPFT to address the mismatch problem while enhancing the quality of the feature extractor. Our experiments demonstrate that FedPFT outperforms state-of-the-art methods by up to 7.08%.
• Score: 12.19025665853089
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In traditional Federated Learning approaches like FedAvg, the global model underperforms when faced with data heterogeneity. Personalized Federated Learning (PFL) enables clients to train personalized models to fit their local data distribution better. However, we surprisingly find that the feature extractor in FedAvg is superior to those in most PFL methods. More interestingly, by applying a linear transformation on local features extracted by the feature extractor to align with the classifier, FedAvg can surpass the majority of PFL methods. This suggests that the primary cause of FedAvg's inadequate performance stems from the mismatch between the locally extracted features and the classifier. While current PFL methods mitigate this issue to some extent, their designs compromise the quality of the feature extractor, thus limiting the full potential of PFL. In this paper, we propose a new PFL framework called FedPFT to address the mismatch problem while enhancing the quality of the feature extractor. FedPFT integrates a feature transformation module, driven by personalized prompts, between the global feature extractor and classifier. In each round, clients first train prompts to transform local features to match the global classifier, followed by training model parameters. This approach can also align the training objectives of clients, reducing the impact of data heterogeneity on model collaboration. Moreover, FedPFT's feature transformation module is highly scalable, allowing for the use of different prompts to tailor local features to various tasks. Leveraging this, we introduce a collaborative contrastive learning task to further refine feature extractor quality. Our experiments demonstrate that FedPFT outperforms state-of-the-art methods by up to 7.08%.

Related papers

• FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning [27.782676760198697]
  Federated Learning (FL) has emerged as a pivotal framework for the development of effective global models. A key challenge in FL is client drift, where data heterogeneity impedes the aggregation of scattered knowledge. We introduce a novel algorithm named FedDr+, which empowers local model alignment using dot-regression loss.
  arXiv  Detail & Related papers  (2024-06-04T14:34:13Z)
• FedImpro: Measuring and Improving Client Update in Federated Learning [77.68805026788836]
  Federated Learning (FL) models often experience client drift caused by heterogeneous data. We present an alternative perspective on client drift and aim to mitigate it by generating improved local models.
  arXiv  Detail & Related papers  (2024-02-10T18:14:57Z)
• Parametric Feature Transfer: One-shot Federated Learning with Foundation Models [14.97955440815159]
  In one-shot federated learning, clients collaboratively train a global model in a single round of communication. This paper introduces FedPFT, a methodology that harnesses the transferability of foundation models to enhance both accuracy and communication efficiency in one-shot FL.
  arXiv  Detail & Related papers  (2024-02-02T19:34:46Z)
• Unlocking the Potential of Prompt-Tuning in Bridging Generalized and Personalized Federated Learning [49.72857433721424]
  Vision Transformers (ViT) and Visual Prompt Tuning (VPT) achieve state-of-the-art performance with improved efficiency in various computer vision tasks. We present a novel algorithm, SGPT, that integrates Generalized FL (GFL) and Personalized FL (PFL) approaches by employing a unique combination of both shared and group-specific prompts.
  arXiv  Detail & Related papers  (2023-10-27T17:22:09Z)
• PFL-GAN: When Client Heterogeneity Meets Generative Models in Personalized Federated Learning [55.930403371398114]
  We propose a novel generative adversarial network (GAN) sharing and aggregation strategy for personalized learning (PFL) PFL-GAN addresses the client heterogeneity in different scenarios. More specially, we first learn the similarity among clients and then develop an weighted collaborative data aggregation. The empirical results through the rigorous experimentation on several well-known datasets demonstrate the effectiveness of PFL-GAN.
  arXiv  Detail & Related papers  (2023-08-23T22:38:35Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learning [9.950367271170592]
  We investigate where and how to partially personalize a Vision Transformers (ViT) model. Based on the insights that the self-attention layer and the classification head are the most sensitive parts of a ViT, we propose a novel approach called FedPerfix. We evaluate the proposed approach on CIFAR-100, OrganAMNIST, and Office-Home datasets and demonstrate its effectiveness compared to several advanced PFL methods.
  arXiv  Detail & Related papers  (2023-08-17T19:22:30Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Personalized Federated Learning with Local Attention [5.018560254008613]
  Federated Learning (FL) aims to learn a single global model that enables the central server to help the model training in local clients without accessing their local data. Key challenge of FL is the heterogeneous label distribution and feature shift, which could lead to significant performance degradation of the learned models. We propose a simple yet effective algorithm, namely textbfpersonalized textbfFederated learning with textbfLocal textbfAttention (pFedLA) Two modules are proposed in pFedLA, i.e., the personalized
  arXiv  Detail & Related papers  (2023-04-02T20:10:32Z)
• Personalized Federated Learning on Long-Tailed Data via Adversarial Feature Augmentation [24.679535905451758]
  PFL aims to learn personalized models for each client based on the knowledge across all clients in a privacy-preserving manner. Existing PFL methods assume that the underlying global data across all clients are uniformly distributed without considering the long-tail distribution. We propose Federated Learning with Adversarial Feature Augmentation (FedAFA) to address this joint problem in PFL.
  arXiv  Detail & Related papers  (2023-03-27T13:00:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.