Decoupled Split Learning via Auxiliary Loss

• URL: http://arxiv.org/abs/2601.19261v1
• Date: Tue, 27 Jan 2026 06:41:42 GMT
• Title: Decoupled Split Learning via Auxiliary Loss
• Abstract summary: Split learning is a distributed training paradigm where a neural network is partitioned between clients and a server.<n>Traditional split learning relies on end-to-end backpropagation across the client-server split point.<n>We develop a beyond-backpropagation training method for split learning.
• Score: 11.21293654556267
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Split learning is a distributed training paradigm where a neural network is partitioned between clients and a server, which allows data to remain at the client while only intermediate activations are shared. Traditional split learning relies on end-to-end backpropagation across the client-server split point. This incurs a large communication overhead (i.e., forward activations and backward gradients need to be exchanged every iteration) and significant memory use (for storing activations and gradients). In this paper, we develop a beyond-backpropagation training method for split learning. In this approach, the client and server train their model partitions semi-independently, using local loss signals instead of propagated gradients. In particular, the client's network is augmented with a small auxiliary classifier at the split point to provide a local error signal, while the server trains on the client's transmitted activations using the true loss function. This decoupling removes the need to send backward gradients, which cuts communication costs roughly in half and also reduces memory overhead (as each side only stores local activations for its own backward pass). We evaluate our approach on CIFAR-10 and CIFAR-100. Our experiments show two key results. First, the proposed approach achieves performance on par with standard split learning that uses backpropagation. Second, it significantly reduces communication (of transmitting activations/gradient) by 50% and peak memory usage by up to 58%.

Related papers

• TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems [0.0]
  We present TL++, a two-mode-learning framework that constructs virtual batches across nodes to recover centralized mini-batch gradient behavior.<n>Secure mode secret-shares each cut-layer activation and gradient between an orchestrator and a non-colluding helper.<n>On CIFAR-10, TL++ base cut 1 and exact secure cut 3 achieve accuracies of 91.41% (SD 0.19) and 90.93% (SD 0.179), respectively, exceeding the strongest measured non-TL++ baseline by more than 12 percentage points.
  arXiv  Detail & Related papers  (2026-06-24T09:34:27Z)
• CycleSL: Server-Client Cyclical Update Driven Scalable Split Learning [60.59553507555341]
  We introduce CycleSL, a novel aggregation-free split learning framework.<n>Inspired by alternating block coordinate descent, CycleSL treats server-side training as an independent higher-level machine learning task.<n>Our empirical findings highlight the effectiveness of CycleSL in enhancing model performance.
  arXiv  Detail & Related papers  (2025-11-23T21:00:21Z)
• FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation [13.034619943612311]
  Collaborative training methods like Federated Learning (FL) and Split Learning (SL) enable distributed machine learning without sharing raw data.<n>We propose FSL-SAGE, a new federated split learning algorithm that estimates server-side gradient feedback via auxiliary models.
  arXiv  Detail & Related papers  (2025-05-29T07:18:59Z)
• An Adaptive Clustering Scheme for Client Selections in Communication-Efficient Federated Learning [3.683202928838613]
  Federated learning is a novel decentralized learning architecture.<n>We propose to dynamically adjust the number of clusters to find the most ideal grouping results.<n>It may reduce the number of users participating in the training to achieve the effect of reducing communication costs without affecting the model performance.
  arXiv  Detail & Related papers  (2025-04-11T08:43:12Z)
• SafeSplit: A Novel Defense Against Client-Side Backdoor Attacks in Split Learning (Full Version) [53.16528046390881]
  Split Learning (SL) is a distributed deep learning approach enabling multiple clients and a server to collaboratively train and infer on a shared deep neural network (DNN)<n>This paper presents SafeSplit, the first defense against client-side backdoor attacks in Split Learning (SL)<n>It uses a two-fold analysis to identify client-induced changes and detect poisoned models.
  arXiv  Detail & Related papers  (2025-01-11T22:20:20Z)
• SplitFedZip: Learned Compression for Data Transfer Reduction in Split-Federated Learning [5.437298646956505]
  SplitFederated (SplitFed) learning is an ideal learning framework across various domains.<n>SplitFedZip is a novel method that employs learned compression to reduce data transfer in SplitFed learning.
  arXiv  Detail & Related papers  (2024-12-18T19:04:19Z)
• Subspace based Federated Unlearning [75.90552823500633]
  Federated unlearning (FL) aims to remove a specified target client's contribution in FL to satisfy the user's right to be forgotten. Most existing federated unlearning algorithms require the server to store the history of the parameter updates. We propose a simple-yet-effective subspace based federated unlearning method, dubbed SFU, that lets the global model perform gradient ascent.
  arXiv  Detail & Related papers  (2023-02-24T04:29:44Z)
• SplitOut: Out-of-the-Box Training-Hijacking Detection in Split Learning via Outlier Detection [0.0]
  Split learning enables efficient and privacy-aware training of a deep neural network by splitting a neural network so that the clients (data holders) compute the first layers and only share the intermediate output with the central compute-heavy server. Server has full control over what the client models learn, which has already been exploited to infer the private data of clients and to implement backdoors in the client models. We show that given modest assumptions regarding the clients' compute capabilities, an out-of-the-box detection method can be used to detect existing training-hijacking attacks with almost-zero false positive rates.
  arXiv  Detail & Related papers  (2023-02-16T23:02:39Z)
• Scalable Collaborative Learning via Representation Sharing [53.047460465980144]
  Federated learning (FL) and Split Learning (SL) are two frameworks that enable collaborative learning while keeping the data private (on device) In FL, each data holder trains a model locally and releases it to a central server for aggregation. In SL, the clients must release individual cut-layer activations (smashed data) to the server and wait for its response (during both inference and back propagation). In this work, we present a novel approach for privacy-preserving machine learning, where the clients collaborate via online knowledge distillation using a contrastive loss.
  arXiv  Detail & Related papers  (2022-11-20T10:49:22Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Communication and Computation Reduction for Split Learning using Asynchronous Training [9.461478650926608]
  This paper proposes a loss-based asynchronous training scheme that updates the client-side model less frequently and only sends/receives activations/gradients in selected epochs. The privacy of the proposed communication reduction based split learning method is almost the same as traditional split learning.
  arXiv  Detail & Related papers  (2021-07-20T22:08:13Z)
• Sparse Communication for Training Deep Networks [56.441077560085475]
  Synchronous gradient descent (SGD) is the most common method used for distributed training of deep learning models. In this algorithm, each worker shares its local gradients with others and updates the parameters using the average gradients of all workers. We study several compression schemes and identify how three key parameters affect the performance.
  arXiv  Detail & Related papers  (2020-09-19T17:28:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.