TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

• URL: http://arxiv.org/abs/2606.25627v1
• Date: Wed, 24 Jun 2026 09:34:27 GMT
• Title: TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems
• Abstract summary: We present TL++, a two-mode-learning framework that constructs virtual batches across nodes to recover centralized mini-batch gradient behavior.<n>Secure mode secret-shares each cut-layer activation and gradient between an orchestrator and a non-colluding helper.<n>On CIFAR-10, TL++ base cut 1 and exact secure cut 3 achieve accuracies of 91.41% (SD 0.19) and 90.93% (SD 0.179), respectively, exceeding the strongest measured non-TL++ baseline by more than 12 percentage points.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Distributed intelligent systems increasingly need to train across data silos without centralizing raw data. Federated learning keeps data local but can suffer under heterogeneous partitions and requires repeated full-model exchange. Split learning reduces communication through cut-layer activations, but standard protocols generally do not recover centralized mini-batch gradient behavior and may expose activations and gradients in plaintext. We present TL++, a two-mode traversal-learning framework that constructs virtual batches across nodes to recover centralized mini-batch gradient behavior under explicit synchronization assumptions. Base mode exchanges cut-layer activations and gradients rather than full models. Secure mode secret-shares each cut-layer activation and gradient between an orchestrator and a non-colluding helper, preventing either server from observing plaintext cut-layer tensors. This protection is limited to a semi-honest two-server setting; labels and loss-related outputs remain visible to the orchestrator. In the lightweight secure path evaluated here, exactness requires a linear or affine server path, while nonlinear operations require nonlinear MPC or approximation. We formalize TL++, analyze communication and computation costs, and evaluate it against federated and split-learning baselines on CIFAR-10 and BioGPT/PubMedQA using full fine-tuning and LoRA. On CIFAR-10, TL++ base cut 1 and exact secure cut 3 achieve accuracies of 91.41% (SD 0.19) and 90.93% (SD 0.17), respectively, exceeding the strongest measured non-TL++ baseline by more than 12 percentage points. TL++ base cut 1 also reduces per-step communication by 13.1-fold relative to full-model synchronization. PubMedQA results similarly favor TL++. Overall, TL++ approaches centralized-training performance while reducing communication and providing activation-level secret sharing.

Related papers

• Replacement Learning: Training Neural Networks with Fewer Parameters [15.09968642484538]
  Replacement Learning (RepL) is a training-time paradigm that reduces full-depth redundancy by replacing selected blocks.<n>RepL preserves local contextual continuity while avoiding unnecessary full-layer.<n>We show that RepL reduces trainable parameters, GPU memory usage, and training time while matching or surpassing standard end-to-end training.
  arXiv  Detail & Related papers  (2026-05-19T08:34:31Z)
• LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections [54.51153206716027]
  Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server.<n>We propose LightSplit, which limits information exposure and reduces communication overhead by applying a lightweight fixed random projection at the cut layer.<n>Our results show that the method retains more than 95% of the baseline accuracy at up to 32x reduction in transmitted dimensionality.
  arXiv  Detail & Related papers  (2026-05-13T09:45:49Z)
• Decoupled Split Learning via Auxiliary Loss [11.21293654556267]
  Split learning is a distributed training paradigm where a neural network is partitioned between clients and a server.<n>Traditional split learning relies on end-to-end backpropagation across the client-server split point.<n>We develop a beyond-backpropagation training method for split learning.
  arXiv  Detail & Related papers  (2026-01-27T06:41:42Z)
• Distributed Training under Packet Loss [8.613477072763404]
  Leveraging unreliable connections will reduce latency but may sacrifice model accuracy and convergence once packets are dropped.<n>We introduce a principled, end-to-end solution that preserves accuracy and convergence guarantees under genuine packet loss.<n>This work bridges the gap between communication-efficient protocols and the accuracy and guarantees demanded by modern large-model training.
  arXiv  Detail & Related papers  (2025-07-02T11:07:20Z)
• Protocol Models: Scaling Decentralized Training with Communication-Efficient Model Parallelism [59.79227116582264]
  Scaling models has led to significant advancements in deep learning, but training these models in decentralized settings remains challenging.<n>We propose a novel compression algorithm that compresses both forward and backward passes, enabling up to 99% compression with no convergence degradation.
  arXiv  Detail & Related papers  (2025-06-02T02:19:22Z)
• Silenzio: Secure Non-Interactive Outsourced MLP Training [5.3909333359654275]
  We introduce Silenuio, the first fully non-interactive outsourcing scheme for the training of inherent ML skills.<n>Silenzio operates entirely over low-bitwidth integer to the computational overhead inherent to FHE.
  arXiv  Detail & Related papers  (2025-04-24T17:59:22Z)
• FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression [55.992528247880685]
  Decentralized training faces significant challenges regarding system design and efficiency. We present FusionLLM, a decentralized training system designed and implemented for training large deep neural networks (DNNs) We show that our system and method can achieve 1.45 - 9.39x speedup compared to baseline methods while ensuring convergence.
  arXiv  Detail & Related papers  (2024-10-16T16:13:19Z)
• Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers [27.253728528979572]
  We consider the problem of gradient learning in the presence of stragglers. We propose a novel DL method based on 1-bit gradient coding (1bit GCDL)
  arXiv  Detail & Related papers  (2024-03-19T06:48:40Z)
• Improved techniques for deterministic l2 robustness [63.34032156196848]
  Training convolutional neural networks (CNNs) with a strict 1-Lipschitz constraint under the $l_2$ norm is useful for adversarial robustness, interpretable gradients and stable training. We introduce a procedure to certify robustness of 1-Lipschitz CNNs by replacing the last linear layer with a 1-hidden layer. We significantly advance the state-of-the-art for standard and provable robust accuracies on CIFAR-10 and CIFAR-100.
  arXiv  Detail & Related papers  (2022-11-15T19:10:12Z)
• Training Recommender Systems at Scale: Communication-Efficient Model and Data Parallelism [56.78673028601739]
  We propose a compression framework called Dynamic Communication Thresholding (DCT) for communication-efficient hybrid training. DCT reduces communication by at least $100times$ and $20times$ during DP and MP, respectively. It improves end-to-end training time for a state-of-the-art industrial recommender model by 37%, without any loss in performance.
  arXiv  Detail & Related papers  (2020-10-18T01:44:42Z)
• Sparse Communication for Training Deep Networks [56.441077560085475]
  Synchronous gradient descent (SGD) is the most common method used for distributed training of deep learning models. In this algorithm, each worker shares its local gradients with others and updates the parameters using the average gradients of all workers. We study several compression schemes and identify how three key parameters affect the performance.
  arXiv  Detail & Related papers  (2020-09-19T17:28:11Z)
• Domain-specific Communication Optimization for Distributed DNN Training [10.781867496460837]
  We present DLCP, a novel solution exploiting the domain-specific properties of deep learning to optimize communication overhead of DNN training in a fine-grained manner. It exploits em bounded loss tolerance of SGD-based training to improve tail communication latency which cannot be avoided purely through gradient compression. It then performs fine-grained packet-level prioritization and dropping, as opposed to flow-level scheduling, based on layers and magnitudes of gradients to further speedup model convergence without affecting accuracy.
  arXiv  Detail & Related papers  (2020-08-16T09:53:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.