GAPSL: A Gradient-Aligned Parallel Split Learning on Heterogeneous Data

• URL: http://arxiv.org/abs/2603.18540v1
• Date: Thu, 19 Mar 2026 06:51:05 GMT
• Title: GAPSL: A Gradient-Aligned Parallel Split Learning on Heterogeneous Data
• Abstract summary: We propose gradient-aligned parallel split learning (PSL) framework that comprises two key components: leader gradient identification (LGI) and gradient direction alignment (GDA)<n> GAPSL consistently outperforms state-of-the-art benchmarks in training accuracy and latency.
• Score: 61.365432566830485
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The increasing complexity of neural networks poses significant challenges for democratizing FL on resource?constrained client devices. Parallel split learning (PSL) has emerged as a promising solution by offloading substantial computing workload to a server via model partitioning, shrinking client-side computing load, and eliminating the client-side model aggregation for reduced communication and deployment costs. Since PSL is aggregation-free, it suffers from severe training divergence stemming from gradient directional inconsistency across clients. To address this challenge, we propose GAPSL, a gradient-aligned PSL framework that comprises two key components: leader gradient identification (LGI) and gradient direction alignment (GDA). LGI dynamically selects a set of directionally consistent client gradients to construct a leader gradient that captures the global convergence trend. GDA employs a direction-aware regularization to align each client's gradient with the leader gradient, thereby mitigating inter-device gradient directional inconsistency and enhancing model convergence. We evaluate GAPSL on a prototype computing testbed. Extensive experiments demonstrate that GAPSL consistently outperforms state-of-the-art benchmarks in training accuracy and latency.

Related papers

• Personalized Federated Learning for Gradient Alignment [8.912327975864768]
  pFLAlign is a gradient alignment framework to maintain client specific information during both local training and aggregation.<n>Our experiments and ablation studies show that pFLAlign consistently improves personalization performance and training stability.
  arXiv  Detail & Related papers  (2026-05-04T01:50:54Z)
• Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization [56.95448807869383]
  Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning.<n>Recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage.<n>We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD)
  arXiv  Detail & Related papers  (2026-04-01T14:32:15Z)
• GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning [55.03441672267886]
  We propose GradAlign, a gradient-aligned data selection method for reinforcement learning.<n>We evaluate GradAlign across three data regimes: unreliable reward signals, distribution imbalance, and low-utility training corpus.
  arXiv  Detail & Related papers  (2026-02-25T01:54:50Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture.<n>Non-smooth regularization is often incorporated into machine learning tasks.<n>We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• Towards Layer-Wise Personalized Federated Learning: Adaptive Layer Disentanglement via Conflicting Gradients [11.269920973751244]
  In personalized Federated Learning (pFL), high data heterogeneity can cause significant gradient divergence across devices. We introduce a new approach to pFL design, namely Federated Learning with Layer-wise Aggregation via Gradient Analysis (FedLAG) FedLAG assigns layers for personalization based on the extent of layer-wise gradient conflicts.
  arXiv  Detail & Related papers  (2024-10-03T14:46:19Z)
• Hierarchical Federated Learning with Multi-Timescale Gradient Correction [24.713834338757195]
  In this paper, we propose a multi-time correction (MTGC) methodology to resolve this issue.<n>Our key idea is to introduce distinct control to (i) correct the client gradient the group gradient, i.e., to reduce client model drift caused by local updates based on individual datasets.
  arXiv  Detail & Related papers  (2024-09-27T05:10:05Z)
• CG-FedLLM: How to Compress Gradients in Federated Fune-tuning for Large Language Models [21.919883617413358]
  This study introduces an innovative approach to compress gradients to improve communication efficiency during Large-Language Models (LLMs) We also present a series of experimental analyses focusing on the signal-to-noise ratio, compression rate, and robustness within this privacy-centric framework.
  arXiv  Detail & Related papers  (2024-05-22T15:32:38Z)
• GIFD: A Generative Gradient Inversion Method with Feature Domain Optimization [52.55628139825667]
  Federated Learning (FL) has emerged as a promising distributed machine learning framework to preserve clients' privacy. Recent studies find that an attacker can invert the shared gradients and recover sensitive data against an FL system by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD), which disassembles the GAN model and searches the feature domains of the intermediate layers.
  arXiv  Detail & Related papers  (2023-08-09T04:34:21Z)
• Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks [44.37047471448793]
  In this paper, we advocate the integration of edge computing paradigm and parallel split learning (PSL) We propose an innovative PSL framework, namely, efficient parallel split learning (EPSL) to accelerate model training. We show that the proposed EPSL framework significantly decreases the training latency needed to achieve a target accuracy.
  arXiv  Detail & Related papers  (2023-03-26T16:09:48Z)
• LASG: Lazily Aggregated Stochastic Gradients for Communication-Efficient Distributed Learning [47.93365664380274]
  This paper targets solving distributed machine learning problems such as federated learning in a communication-efficient fashion. A class of new gradient descent (SGD) approaches have been developed, which can be viewed as a generalization to the recently developed lazily aggregated gradient (LAG) method. The key components of LASG are a set of new rules tailored for gradients that can be implemented either to save download, upload, or both.
  arXiv  Detail & Related papers  (2020-02-26T08:58:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.