Personalized Federated Learning for Gradient Alignment

• URL: http://arxiv.org/abs/2605.02143v1
• Date: Mon, 04 May 2026 01:50:54 GMT
• Title: Personalized Federated Learning for Gradient Alignment
• Abstract summary: pFLAlign is a gradient alignment framework to maintain client specific information during both local training and aggregation.<n>Our experiments and ablation studies show that pFLAlign consistently improves personalization performance and training stability.
• Score: 8.912327975864768
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Personalized federated learning (pFL) aims to adapt models to client specific data distributions, yet it often fails to reliably preserve personalized information. Local training is hindered by high variance gradients induced by limited and heterogeneous client data, while aggregation further distorts client specific optimization directions. To address these challenges, we propose pFLAlign, a gradient alignment framework to maintain client specific information during both local training and aggregation. pFLAlign consists of two complementary mechanisms: one adapts local gradient directions to reduce variance during client side optimization, and the other mitigates aggregation induced distortion by realigning the global model with each client's personalized direction. Theoretically, we derive pFLAlign from a PAC Bayesian analysis, which reveals how personalized gradient alignment preserves client specific information. Our experiments and ablation studies show that pFLAlign consistently improves personalization performance and training stability, achieving state of the art results.

Related papers

• GAPSL: A Gradient-Aligned Parallel Split Learning on Heterogeneous Data [61.365432566830485]
  We propose gradient-aligned parallel split learning (PSL) framework that comprises two key components: leader gradient identification (LGI) and gradient direction alignment (GDA)<n> GAPSL consistently outperforms state-of-the-art benchmarks in training accuracy and latency.
  arXiv  Detail & Related papers  (2026-03-19T06:51:05Z)
• CO-PFL: Contribution-Oriented Personalized Federated Learning for Heterogeneous Networks [51.43780477302533]
  Contribution-Oriented PFL (CO-PFL) is a novel algorithm that dynamically estimates each client's contribution for global aggregation.<n>CO-PFL consistently surpasses state-of-the-art methods in robustness in personalization accuracy, robustness, scalability and convergence stability.
  arXiv  Detail & Related papers  (2025-10-23T05:10:06Z)
• FedPPA: Progressive Parameter Alignment for Personalized Federated Learning [0.9931624906346306]
  Federated Learning (FL) is designed as a decentralized, privacy-preserving machine learning paradigm that enables multiple clients to collaboratively train a model without sharing their data.<n>In real-world scenarios, clients often have heterogeneous computational resources and hold non-independent and identically distributed data (non-IID), which poses significant challenges during training.<n>We propose Progressive Alignment (FedPPA), which progressively aligns the weights of common layers across clients with the global model's weights.<n>Experiments on three image classification datasets, including MNIST, FMNIST, and CIFAR-10, demonstrate that FedPPA consistently outperforms existing FL algorithms,
  arXiv  Detail & Related papers  (2025-10-16T14:03:05Z)
• FedHiP: Heterogeneity-Invariant Personalized Federated Learning Through Closed-Form Solutions [22.424881385529783]
  We propose a Heterogeneity-invariant Personalized Federated learning scheme, named FedHiP, through analytical (i.e., closed-form) solutions to avoid gradient-based updates.<n>Our scheme incorporates three phases: analytic local training, analytic global aggregation, and analytic local personalization.<n>Experiments on benchmark datasets validate the superiority of our FedHiP scheme, outperforming the state-of-the-art baselines by at least 5.79%-20.97% in accuracy.
  arXiv  Detail & Related papers  (2025-08-06T14:15:57Z)
• FedAli: Personalized Federated Learning Alignment with Prototype Layers for Generalized Mobile Services [9.683642138601464]
  Federated Alignment (FedAli) is a prototype-based regularization technique that enhances inter-client alignment while strengthening the robustness of personalized adaptations.<n>At its core, FedAli introduces the ALignment with Prototypes layer, inspired by human memory, to enhance generalization.<n>Our experiments show that FedAli significantly enhances client generalization while preserving strong personalization in heterogeneous settings.
  arXiv  Detail & Related papers  (2024-11-15T21:35:21Z)
• FedImpro: Measuring and Improving Client Update in Federated Learning [77.68805026788836]
  Federated Learning (FL) models often experience client drift caused by heterogeneous data. We present an alternative perspective on client drift and aim to mitigate it by generating improved local models.
  arXiv  Detail & Related papers  (2024-02-10T18:14:57Z)
• Adaptive Self-Distillation for Minimizing Client Drift in Heterogeneous Federated Learning [9.975023463908496]
  Federated Learning (FL) is a machine learning paradigm that enables clients to jointly train a global model by aggregating the locally trained models without sharing any local training data. We propose a novel regularization technique based on adaptive self-distillation (ASD) for training models on the client side. Our regularization scheme adaptively adjusts to the client's training data based on the global model entropy and the client's label distribution.
  arXiv  Detail & Related papers  (2023-05-31T07:00:42Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Visual Prompt Based Personalized Federated Learning [83.04104655903846]
  We propose a novel PFL framework for image classification tasks, dubbed pFedPT, that leverages personalized visual prompts to implicitly represent local data distribution information of clients. Experiments on the CIFAR10 and CIFAR100 datasets show that pFedPT outperforms several state-of-the-art (SOTA) PFL algorithms by a large margin in various settings.
  arXiv  Detail & Related papers  (2023-03-15T15:02:15Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.