CloudCons: A Comprehensive End-to-End Benchmark for Cloud Resource Consolidation

• URL: http://arxiv.org/abs/2606.13513v1
• Date: Thu, 11 Jun 2026 16:00:51 GMT
• Title: CloudCons: A Comprehensive End-to-End Benchmark for Cloud Resource Consolidation
• Abstract summary: We propose CloudCons, a benchmark designed to evaluate forecasting models within the context of cloud resource consolidation.<n>We build high-quality datasets that cover diverse workloads from Huawei Cloud, Microsoft, Azure, and Google Borg.<n>Our experiments reveal a pivotal finding: while foundation models demonstrate superior zero-shot forecasting accuracy, this advantage does not inherently translate into better decision utility.
• Score: 33.524558997797136
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Driven by conservative over-provisioning to guarantee service reliability, resource utilization in cloud data centers remains at low levels. To mitigate this, the forecast-then-optimize paradigm has emerged to optimize consolidation by anticipating future demands. While emerging time series foundation models promise to enhance this paradigm through zero-shot generalization, existing benchmarks focus solely on prediction error metrics. The actual decision utility of these advanced models remains unverified, rendering their practical value for downstream tasks uncertain. To bridge this gap, we propose CloudCons, a comprehensive end-to-end benchmark designed to evaluate forecasting models within the specific context of cloud resource consolidation. We build high-quality datasets that cover diverse workloads from Huawei Cloud, Microsoft Azure, and Google Borg, capturing distinct service characteristics ranging from synchronized diurnal rhythms to stochastic, pulse-like bursts and high-frequency noise. We conduct an extensive evaluation of statistical, deep learning, and foundation models. Our experiments reveal a pivotal finding: while foundation models demonstrate superior zero-shot forecasting accuracy, this advantage does not inherently translate into better decision utility. Of practical significance, we systematically analyze how the selection of predictive quantiles acts as a critical lever. We provide actionable guidelines for calibrating these selections to balance the trade-off between resource efficiency and service reliability, offering vital insights for real-world deployment decisions.

Related papers

• A Two-Stage Forecasting System for CPU Workload Prediction in Private Clouds [0.4202074945058886]
  This study proposes a two-stage integrated forecasting model that explicitly models this dependency by first forecasting customer service requests.<n>The proposed work was evaluated using real-world traces collected from a private cloud environment comprising ten applications.
  arXiv  Detail & Related papers  (2026-09-03T07:15:44Z)
• ScaleSense: Cost-Intelligent Scaling Framework via Learned Resource Estimation in Alibaba AnalyticDB [59.708423132520515]
  Cloud-native serverless data warehouses achieve fine-grained elasticity by decoupling storage from compute.<n>Yet determining the optimal resource allocation for highly heterogeneous ad-hoc queries remains a formidable industrial challenge.<n>We propose ScaleSense, a proactive, query-level resource scaling framework.
  arXiv  Detail & Related papers  (2026-08-08T06:10:35Z)
• A Novel Approach to Temporal QoS Estimation via Extended Kalman Filter-Incorporated Latent Feature Analysis [20.92363732107246]
  We propose a novel Extended Kalman Filter-Enhanced Latent Feature Analysis (EKL) model to perform efficient and accurate temporal prediction.<n>Our proposed EKL surpasses existing state-of-the-art models with respect to both computational efficiency and prediction accuracy for missing temporal data.<n> Experimental evaluations conducted on real-world temporal datasets reveal that our proposed EKL surpasses existing state-of-the-art models with respect to both computational efficiency and prediction accuracy for missing temporal data.
  arXiv  Detail & Related papers  (2026-06-22T08:22:18Z)
• Conformal Reliability: A New Evaluation Metric for Conditional Generation [62.761166941396844]
  Conditional generative models have recently remarkable success in various applications.<n>We propose a novel evaluation metric called reliability based on prediction set at a pre-specified confidence level.<n>We introduce Conformal ReLiability (CReL), a framework that can (i) construct a prediction set and (ii) accurately optimize the reliability score within the constructed prediction set.
  arXiv  Detail & Related papers  (2026-05-29T03:52:44Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• QoSBERT: An Uncertainty-Aware Approach based on Pre-trained Language Models for Service Quality Prediction [4.711507071955676]
  We propose BERT, the first framework that reformulates prediction as a semantic regression task based on pre trained language models.<n>We integrate a Monte Carlo Dropout based uncertainty estimation module, allowing for trustworthy risk-aware service quality prediction.<n>BERT achieves an average reduction of 11.7% in MAE and 6.7% in RMSE for response time prediction, and 6.9% in MAE for throughput prediction compared to the strongest baselines.
  arXiv  Detail & Related papers  (2025-05-09T03:15:45Z)
• Decision-Focused Fine-Tuning of Time Series Foundation Models for Dispatchable Feeder Optimization [0.5808168734833972]
  We use decision-focused fine-tuning within time series foundation models to offer a scalable and efficient solution for the dispatchable feeder optimization problem.<n>To obtain more robust predictions for scarce building data, we use Moirai as a state-of-the-art foundation model.<n>Comparing the decision-focused fine-tuned Moirai with a state-of-the-art classical prediction-focused fine-tuning Morai, we observe an improvement of 9.45% in average total daily costs.
  arXiv  Detail & Related papers  (2025-03-03T07:47:20Z)
• Context is Key: A Benchmark for Forecasting with Essential Textual Information [87.3175915185287]
  "Context is Key" (CiK) is a forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context.<n>We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters.<n>We propose a simple yet effective LLM prompting method that outperforms all other tested methods on our benchmark.
  arXiv  Detail & Related papers  (2024-10-24T17:56:08Z)
• Share Secrets for Privacy: Confidential Forecasting with Vertical Federated Learning [5.584904689846748]
  Key challenges include data privacy and over-fitting on small and noisy datasets during both training and inference.<n>We propose Secret-shared Time Series Forecasting with VFL'' (STV), a novel framework with the following key features.<n>Results demonstrate that STV's forecasting accuracy is comparable to those of centralized approaches.
  arXiv  Detail & Related papers  (2024-05-31T12:27:38Z)
• Uncertainty-boosted Robust Video Activity Anticipation [72.14155465769201]
  Video activity anticipation aims to predict what will happen in the future, embracing a broad application prospect ranging from robot vision to autonomous driving. Despite the recent progress, the data uncertainty issue, reflected as the content evolution process and dynamic correlation in event labels, has been somehow ignored. We propose an uncertainty-boosted robust video activity anticipation framework, which generates uncertainty values to indicate the credibility of the anticipation results.
  arXiv  Detail & Related papers  (2024-04-29T12:31:38Z)
• Streaming Motion Forecasting for Autonomous Driving [71.7468645504988]
  We introduce a benchmark that queries future trajectories on streaming data and we refer to it as "streaming forecasting" Our benchmark inherently captures the disappearance and re-appearance of agents, which is a safety-critical problem yet overlooked by snapshot-based benchmarks. We propose a plug-and-play meta-algorithm called "Predictive Streamer" that can adapt any snapshot-based forecaster into a streaming forecaster.
  arXiv  Detail & Related papers  (2023-10-02T17:13:16Z)
• Forecasting Workload in Cloud Computing: Towards Uncertainty-Aware Predictions and Transfer Learning [1.5749416770494704]
  We show that modelling the uncertainty of predictions has a positive impact on performance. We investigate whether our models benefit transfer learning capabilities across different domains.
  arXiv  Detail & Related papers  (2023-02-24T14:51:30Z)
• Trust-Based Cloud Machine Learning Model Selection For Industrial IoT and Smart City Services [5.333802479607541]
  We consider the paradigm where cloud service providers collect big data from resource-constrained devices for building Machine Learning prediction models. Our proposed solution comprises an intelligent-time reconfiguration that maximizes the level of trust of ML models. Our results show that the selected model's trust level is 0.7% to 2.53% less compared to the results obtained using ILP.
  arXiv  Detail & Related papers  (2020-08-11T23:58:03Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.