Accelerating Divisible Load Processing Through Machine Learning: A Practical Framework for Large-Scale Workloads

• URL: http://arxiv.org/abs/2605.23247v1
• Date: Fri, 22 May 2026 05:31:15 GMT
• Title: Accelerating Divisible Load Processing Through Machine Learning: A Practical Framework for Large-Scale Workloads
• Abstract summary: We introduce the first machine learning framework for predicting optimal processing times in Single-Level Tree Network (SLTN) architectures.<n>We train a model on 100,000 synthetically generated configurations to predict optimal processing times without explicit formulation of Divisible Load Theory equations.<n>The model achieves 97-99% accuracy (R-square factor) with mean absolute percentage error of 1-5%, demonstrating that neural networks can effectively learn complex load distribution relationships.
• Score: 6.459138923461498
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we introduce the first machine learning framework for predicting optimal processing times in Single-Level Tree Network (SLTN) architectures for the Divisible Load Theory (DLT) paradigm. Using a feedforward neural network(FNN) with 16 engineered features, we train a model on 100,000 synthetically generated configurations to predict optimal processing times without explicit formulation of DLT equations. The model achieves 97-99% accuracy (R-square factor) with mean absolute percentage error of 1-5%, demonstrating that neural networks can effectively learn complex load distribution relationships. Feature importance analysis reveals that the model implicitly captures DLT mathematical structure, including load conservation and simultaneous finishing constraints. With inference times under 1 millisecond, the approach provides 10-100x speedup over traditional DLT computation, enabling applications in real-time scheduling, design space exploration, and cloud resource allocation. The method generalizes well across diverse system configurations (n=3 to 20, load size =1 to 100 GB) with consistent accuracy, though performance degrades slightly for very large or highly heterogeneous systems. This work demonstrates the feasibility of using machine learning to accelerate distributed computing optimization while maintaining near-optimal accuracy.

Related papers

• Model-Free Surrogate-Assisted Neural Architecture Search for Evolving Variable-Length Dense Blocks [2.473241744699909]
  We propose a Model-Free Surrogate PSO Network (MFSPNet) for evolving convolutional neural network architectures.<n>MFSPNet integrates a lightweight model-free surrogate predictor within a particle swarm optimization (PSO) framework, eliminating the need for pre-trained surrogate models.<n>Under a consistent training protocol with ten independent runs, the proposed method attains error rates of 3.91%, 17.68%, and 1.91% on CIFAR-10, CIFAR-100, and SVHN, respectively.
  arXiv  Detail & Related papers  (2026-09-02T11:32:07Z)
• Deep Hierarchical Learning with Nested Subspace Networks [53.71337604556311]
  We propose Nested Subspace Networks (NSNs) for large neural networks.<n>NSNs enable a single model to be dynamically and granularly adjusted across a continuous spectrum of compute budgets.<n>We show that NSNs can be surgically applied to pre-trained LLMs and unlock a smooth and predictable compute-performance frontier.
  arXiv  Detail & Related papers  (2025-09-22T15:13:14Z)
• The Eclipsing Binaries via Artificial Intelligence. II. Need for Speed in PHOEBE Forward Models [0.0]
  In modern astronomy, the quantity of data collected has vastly exceeded the capacity for manual analysis.<n> AI can optimize simulation codes where computational bottlenecks arise from the time required to generate forward models.<n>We present a fully connected feedforward artificial neural network (ANN) trained on a dataset of over one million synthetic light curves.
  arXiv  Detail & Related papers  (2024-12-16T15:00:15Z)
• Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch [72.26822499434446]
  Auto-Train-Once (ATO) is an innovative network pruning algorithm designed to automatically reduce the computational and storage costs of DNNs. We provide a comprehensive convergence analysis as well as extensive experiments, and the results show that our approach achieves state-of-the-art performance across various model architectures.
  arXiv  Detail & Related papers  (2024-03-21T02:33:37Z)
• Symbolic Regression on FPGAs for Fast Machine Learning Inference [2.0920303420933273]
  High-energy physics community is investigating the potential of deploying machine-learning-based solutions on Field-Programmable Gate Arrays (FPGAs) We introduce a novel end-to-end procedure that utilizes a machine learning technique called symbolic regression (SR) We show that our approach can approximate a 3-layer neural network using an inference model that achieves up to a 13-fold decrease in execution time, down to 5 ns, while still preserving more than 90% approximation accuracy.
  arXiv  Detail & Related papers  (2023-05-06T17:04:02Z)
• Partitioning Distributed Compute Jobs with Reinforcement Learning and Graph Neural Networks [58.720142291102135]
  Large-scale machine learning models are bringing advances to a broad range of fields. Many of these models are too large to be trained on a single machine, and must be distributed across multiple devices. We show that maximum parallelisation is sub-optimal in relation to user-critical metrics such as throughput and blocking rate.
  arXiv  Detail & Related papers  (2023-01-31T17:41:07Z)
• Unifying Synergies between Self-supervised Learning and Dynamic Computation [53.66628188936682]
  We present a novel perspective on the interplay between SSL and DC paradigms. We show that it is feasible to simultaneously learn a dense and gated sub-network from scratch in a SSL setting. The co-evolution during pre-training of both dense and gated encoder offers a good accuracy-efficiency trade-off.
  arXiv  Detail & Related papers  (2023-01-22T17:12:58Z)
• FINETUNA: Fine-tuning Accelerated Molecular Simulations [5.543169726358164]
  We present an online active learning framework for accelerating the simulation of atomic systems efficiently and accurately. A method of transfer learning to incorporate prior information from pre-trained models accelerates simulations by reducing the number of DFT calculations by 91%. Experiments on 30 benchmark adsorbate-catalyst systems show that our method of transfer learning to incorporate prior information from pre-trained models accelerates simulations by reducing the number of DFT calculations by 91%.
  arXiv  Detail & Related papers  (2022-05-02T21:36:01Z)
• LCS: Learning Compressible Subspaces for Adaptive Network Compression at Inference Time [57.52251547365967]
  We propose a method for training a "compressible subspace" of neural networks that contains a fine-grained spectrum of models. We present results for achieving arbitrarily fine-grained accuracy-efficiency trade-offs at inference time for structured and unstructured sparsity. Our algorithm extends to quantization at variable bit widths, achieving accuracy on par with individually trained networks.
  arXiv  Detail & Related papers  (2021-10-08T17:03:34Z)
• Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch [75.69506249886622]
  Sparsity in Deep Neural Networks (DNNs) has been widely studied to compress and accelerate the models on resource-constrained environments. In this paper, we are the first to study training from scratch an N:M fine-grained structured sparse network.
  arXiv  Detail & Related papers  (2021-02-08T05:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.