Duration-Informed Workload Scheduler

• URL: http://arxiv.org/abs/2604.09599v1
• Date: Sat, 07 Mar 2026 17:53:18 GMT
• Title: Duration-Informed Workload Scheduler
• Abstract summary: We develop a workload scheduler enhanced with a duration prediction module built via Machine Learning.<n>We show its effectiveness using workload traces from a Tier-0 supercomputer, demonstrating a decrease in mean waiting time across all jobs of around 11%.<n>Lower waiting times are directly connected to better quality of service from the users' point of view and higher turnaround from the system's perspective.
• Score: 2.0263791972068628
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-performance computing systems are complex machines whose behaviour is governed by the correct functioning of its many subsystems. Among these, the workload scheduler has a crucial impact on the timely execution of the jobs continuously submitted to the computing resources. Making high-quality scheduling decisions is contingent on knowing the duration of submitted jobs before their execution--a non-trivial task for users that can be tackled with Machine Learning. In this work, we devise a workload scheduler enhanced with a duration prediction module built via Machine Learning. We evaluate its effectiveness and show its performance using workload traces from a Tier-0 supercomputer, demonstrating a decrease in mean waiting time across all jobs of around 11%. Lower waiting times are directly connected to better quality of service from the users' point of view and higher turnaround from the system's perspective.

Related papers

• Semantic-Aware Scheduling for GPU Clusters with Large Language Models [60.14838697778884]
  We propose SchedMate, a framework that bridges the semantic gap between schedulers and jobs they manage.<n>SchedMate extracts deep insights from overlooked, unstructured data sources: source code, runtime logs, and historical jobs.<n>We show SchedMate reduces average job completion times by up to 1.91x, substantially enhancing the scheduling performance.
  arXiv  Detail & Related papers  (2025-10-02T02:01:02Z)
• Robust Speech-Workload Estimation for Intelligent Human-Robot Systems [0.0]
  Intelligent modulation of the system's demands and interaction modality in response to changes in operator workload state may increase performance by avoiding undesirable workload states.<n>Existing workload systems estimate multiple workload components post-hoc, but few estimate speech workload, or function in real-time.<n>An algorithm to estimate speech workload and undesirable workload states in real-time is presented.
  arXiv  Detail & Related papers  (2025-07-08T13:41:59Z)
• ATA: Adaptive Task Allocation for Efficient Resource Management in Distributed Machine Learning [54.08906841213777]
  Asynchronous methods are fundamental for parallelizing computations in distributed machine learning.<n>We propose ATA (Adaptive Task Allocation), a method that adapts to heterogeneous and random distributions of computation times.<n>We show that ATA identifies the optimal task allocation and performs comparably to methods with prior knowledge of computation times.
  arXiv  Detail & Related papers  (2025-02-02T12:22:26Z)
• An Advanced Reinforcement Learning Framework for Online Scheduling of Deferrable Workloads in Cloud Computing [37.457951933256055]
  We propose an online deferrable job scheduling method called textitOnline Scheduling for DEferrable jobs in Cloud (OSDEC), where a deep reinforcement learning model is adopted to learn the scheduling policy. The proposed method can well plan the deployment schedule and achieve a short waiting time for users while maintaining a high resource utilization for the platform.
  arXiv  Detail & Related papers  (2024-06-03T06:55:26Z)
• Third-Party Language Model Performance Prediction from Instruction [59.574169249307054]
  Language model-based instruction-following systems have lately shown increasing performance on many benchmark tasks. A user may easily prompt a model with an instruction without any idea of whether the responses should be expected to be accurate. We propose a third party performance prediction framework, where a separate model is trained to predict the metric resulting from evaluating an instruction-following system on a task.
  arXiv  Detail & Related papers  (2024-03-19T03:53:47Z)
• Acela: Predictable Datacenter-level Maintenance Job Scheduling [27.990173338574138]
  We present Acela, a machine learning system for predicting maintenance job duration. We show that Acela reduces the number of servers that are taken offline by 1.87-4.28X, and reduces the server offline time by 1.40-2.80X.
  arXiv  Detail & Related papers  (2022-12-10T00:22:49Z)
• Music Instrument Classification Reprogrammed [79.68916470119743]
  "Reprogramming" is a technique that utilizes pre-trained deep and complex neural networks originally targeting a different task by modifying and mapping both the input and output of the pre-trained model. We demonstrate that reprogramming can effectively leverage the power of the representation learned for a different task and that the resulting reprogrammed system can perform on par or even outperform state-of-the-art systems at a fraction of training parameters.
  arXiv  Detail & Related papers  (2022-11-15T18:26:01Z)
• Lifelong Learning Metrics [63.8376359764052]
  The DARPA Lifelong Learning Machines (L2M) program seeks to yield advances in artificial intelligence (AI) systems. This document outlines a formalism for constructing and characterizing the performance of agents performing lifelong learning scenarios.
  arXiv  Detail & Related papers  (2022-01-20T16:29:14Z)
• Scheduling Real-time Deep Learning Services as Imprecise Computations [11.611969843191433]
  The paper presents an efficient real-time scheduling algorithm for intelligent real-time edge services. These services perform machine intelligence tasks, such as voice recognition, LIDAR processing, or machine vision. We show that deep neural network can be cast as imprecise computations, each with a mandatory part and several optional parts.
  arXiv  Detail & Related papers  (2020-11-02T16:43:04Z)
• Temporally Correlated Task Scheduling for Sequence Learning [143.70523777803723]
  In many applications, a sequence learning task is usually associated with multiple temporally correlated auxiliary tasks. We introduce a learnable scheduler to sequence learning, which can adaptively select auxiliary tasks for training. Our method significantly improves the performance of simultaneous machine translation and stock trend forecasting.
  arXiv  Detail & Related papers  (2020-07-10T10:28:54Z)
• Sequence-to-sequence models for workload interference [1.988145627448243]
  Co-scheduling of jobs in data-centers is a challenging scenario, where jobs can compete for resources yielding to severe slowdowns or failed executions. Current techniques, most of them already involving machine learning and job modeling, are based on workload behavior summarization across time. We propose a methodology for modeling co-scheduling of jobs on data-centers, based on their behavior towards resources and execution time.
  arXiv  Detail & Related papers  (2020-06-25T14:11:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.