Leakage-Safe and Scheduler-Aware Machine Learning for Grid Job Runtime Prediction

• URL: http://arxiv.org/abs/2609.13701v1
• Date: Sat, 12 Sep 2026 04:20:41 GMT
• Title: Leakage-Safe and Scheduler-Aware Machine Learning for Grid Job Runtime Prediction
• Abstract summary: This paper revisits CPU burst time prediction on the GWA-T-4 AuverGrid workload trace and reformulates it as leakage-safe pre-execution job runtime prediction.<n>We define the target as job-level runtime, use only submission-time attributes, exclude post-execution variables, and evaluate models under temporal and cold-start settings.
• Score: 0.42821598129654453
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate job runtime prediction can improve scheduling-aware resource management in grid and distributed computing environments, but prediction models must be evaluated under realistic deployment constraints. This paper revisits CPU burst time prediction on the GWA-T-4 AuverGrid workload trace and reformulates it as leakage-safe pre-execution job runtime prediction. We define the target as job-level runtime, use only submission-time attributes, exclude post-execution variables, and evaluate models under temporal and cold-start settings rather than relying only on random cross-validation. We compare standard regressors, chronological historical baselines, categorical encoding strategies, and CatBoost with native categorical handling. We further add temporal hyperparameter tuning, runtime predictability analysis, feature ablation, error analysis by job length, and a minimal scheduling simulation. After temporal-validation tuning, CatBoost achieves the strongest deployment-oriented result with R^2=0.239, MAE=27,019, RMSE=46,587, and LogMAE=2.646 on the held-out temporal test set. A single-server simulation over all 69,523 held-out temporal test jobs shows that prediction-informed SJF reduces average waiting time by 50.92% relative to FCFS. The results show that random-split evaluation overestimates performance, categorical-native boosting improves temporal generalization, and long-job underestimation remains a scheduler-relevant challenge.

Related papers

• CastFlow: Learning Role-Specialized Agentic Workflows for Time Series Forecasting [74.29761960394848]
  We propose CastFlow, a dynamic agentic forecasting framework for large language models (LLMs)<n>CastFlow organizes the forecasting process into planning, action, forecasting, and reflection, establishing an agentic workflow.<n>Third, CastFlow adopts a role-specialized design that combines general-purpose reasoning with specialized numerical forecasting.
  arXiv  Detail & Related papers  (2026-04-30T13:24:42Z)
• Prescriptive Scaling Reveals the Evolution of Language Model Capabilities [22.14002750185524]
  We estimate capability boundaries, high conditional quantiles of benchmark scores as a function of log pre training FLOPs.<n>We validate the temporal reliability by fitting on earlier model generations and evaluating on later releases.<n>We introduce an efficient algorithm that recovers near full data frontiers using roughly 20% of evaluation budget.
  arXiv  Detail & Related papers  (2026-02-17T03:13:51Z)
• Echo State Networks for Time Series Forecasting: Hyperparameter Sweep and Benchmarking [51.56484100374058]
  We evaluate whether a fully automatic, purely feedback-driven ESN can serve as a competitive alternative to widely used statistical forecasting methods.<n>Forecast accuracy is measured using MASE and sMAPE and benchmarked against simple benchmarks like drift and seasonal naive and statistical models.
  arXiv  Detail & Related papers  (2026-02-03T16:01:22Z)
• SARIMAX-Based Power Outage Prediction During Extreme Weather Events [9.853802100348892]
  This study develops a SARIMAX-based prediction system for short-term power outage forecasting during extreme weather events.<n>Using hourly data from Michigan counties with outage counts and comprehensive weather features, we implement a systematic two-stage feature engineering pipeline.<n>The selected features are augmented with temporal embeddings, multi-scale lag features, and weather variables with their corresponding lags as inputs to the SARIMAX model.
  arXiv  Detail & Related papers  (2025-11-02T17:13:58Z)
• Enhancing Transformer-Based Foundation Models for Time Series Forecasting via Bagging, Boosting and Statistical Ensembles [7.787518725874443]
  Time series foundation models (TSFMs) have shown strong generalization and zero-shot capabilities for time series forecasting, anomaly detection, classification, and imputation.<n>This paper investigates a suite of statistical and ensemble-based enhancement techniques to improve robustness and accuracy.
  arXiv  Detail & Related papers  (2025-08-18T04:06:26Z)
• Rethinking Remaining Useful Life Prediction with Scarce Time Series Data: Regression under Indirect Supervision [4.335413713700667]
  We introduce a unified framework called parameterized static regression, which takes single points as inputs for regression of target values.<n>Our method demonstrates competitive performance in prediction accuracy when dealing with highly scarce time series data.
  arXiv  Detail & Related papers  (2025-04-12T13:14:35Z)
• SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process [76.98721879039559]
  We propose SMURF-THP, a score-based method for learning Transformer Hawkes process and quantifying prediction uncertainty. Specifically, SMURF-THP learns the score function of events' arrival time based on a score-matching objective. We conduct extensive experiments in both event type prediction and uncertainty quantification of arrival time.
  arXiv  Detail & Related papers  (2023-10-25T03:33:45Z)
• Balancing Computational Efficiency and Forecast Error in Machine Learning-based Time-Series Forecasting: Insights from Live Experiments on Meteorological Nowcasting [0.0]
  This paper addresses the relationship between computational cost and forecast error using meteorological nowcasting. During a 5-day live experiment, 4000 data sources were streamed for training and inferencing 144 models per hour. Results indicate that using the Variance Horizon reduced computational usage by more than 50%, while increasing between 0-15% in error.
  arXiv  Detail & Related papers  (2023-09-26T19:10:00Z)
• Efficient and Differentiable Conformal Prediction with General Function Classes [96.74055810115456]
  We propose a generalization of conformal prediction to multiple learnable parameters. We show that it achieves approximate valid population coverage and near-optimal efficiency within class. Experiments show that our algorithm is able to learn valid prediction sets and improve the efficiency significantly.
  arXiv  Detail & Related papers  (2022-02-22T18:37:23Z)
• Improving Event Duration Prediction via Time-aware Pre-training [90.74988936678723]
  We introduce two effective models for duration prediction. One model predicts the range/unit where the duration value falls in (R-pred); and the other predicts the exact duration value E-pred. Our best model -- E-pred, substantially outperforms previous work, and captures duration information more accurately than R-pred.
  arXiv  Detail & Related papers  (2020-11-05T01:52:11Z)
• AutoCP: Automated Pipelines for Accurate Prediction Intervals [84.16181066107984]
  This paper proposes an AutoML framework called Automatic Machine Learning for Conformal Prediction (AutoCP) Unlike the familiar AutoML frameworks that attempt to select the best prediction model, AutoCP constructs prediction intervals that achieve the user-specified target coverage rate. We tested AutoCP on a variety of datasets and found that it significantly outperforms benchmark algorithms.
  arXiv  Detail & Related papers  (2020-06-24T23:13:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.