Ksurf-Drone: Attention Kalman Filter for Contextual Bandit Optimization in Cloud Resource Allocation

• URL: http://arxiv.org/abs/2511.09766v1
• Date: Fri, 14 Nov 2025 01:08:42 GMT
• Title: Ksurf-Drone: Attention Kalman Filter for Contextual Bandit Optimization in Cloud Resource Allocation
• Abstract summary: Ksurf is a state-of-the-art variance-minimizing estimator method ideal for highly variable cloud data.<n>This work evaluates the performance of Ksurf on estimation-based resource orchestration tasks involving highly variable workloads.<n>Ksurf enables significantly lower latency variance of $41%$ at p95 and $47%$ at p99, demonstrating a $4%$ reduction in CPU usage and 7 MB reduction in master node memory usage.
• Score: 6.651912879528297
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Resource orchestration and configuration parameter search are key concerns for container-based infrastructure in cloud data centers. Large configuration search space and cloud uncertainties are often mitigated using contextual bandit techniques for resource orchestration including the state-of-the-art Drone orchestrator. Complexity in the cloud provider environment due to varying numbers of virtual machines introduces variability in workloads and resource metrics, making orchestration decisions less accurate due to increased nonlinearity and noise. Ksurf, a state-of-the-art variance-minimizing estimator method ideal for highly variable cloud data, enables optimal resource estimation under conditions of high cloud variability. This work evaluates the performance of Ksurf on estimation-based resource orchestration tasks involving highly variable workloads when employed as a contextual multi-armed bandit objective function model for cloud scenarios using Drone. Ksurf enables significantly lower latency variance of $41\%$ at p95 and $47\%$ at p99, demonstrates a $4\%$ reduction in CPU usage and 7 MB reduction in master node memory usage on Kubernetes, resulting in a $7\%$ cost savings in average worker pod count on VarBench Kubernetes benchmark.

Related papers

• ScaleSense: Cost-Intelligent Scaling Framework via Learned Resource Estimation in Alibaba AnalyticDB [59.708423132520515]
  Cloud-native serverless data warehouses achieve fine-grained elasticity by decoupling storage from compute.<n>Yet determining the optimal resource allocation for highly heterogeneous ad-hoc queries remains a formidable industrial challenge.<n>We propose ScaleSense, a proactive, query-level resource scaling framework.
  arXiv  Detail & Related papers  (2026-08-08T06:10:35Z)
• DYNA-PRUNER: Input-Adaptive Data-Model Co-Pruning for Efficient and Scalable Spatio-Temporal Media Prediction [24.16400471463046]
  Dyna-Pruner is an end-to-end framework for input-dependent co-pruning of data and model structure.<n>A shared-importance synchronization mechanism generates coupled masks that prune redundant regions.<n>Experiments on WeatherBench, SEVIR, and TaxiBJ show seamless integration with CNN, RNN, and Transformer backbones.
  arXiv  Detail & Related papers  (2026-06-13T15:14:05Z)
• Speculative Policy Orchestration: A Latency-Resilient Framework for Cloud-Robotic Manipulation [6.709418204725671]
  Cloud robotics enables robots to offload high-dimensional motion planning and reasoning to remote servers.<n>For continuous manipulation tasks requiring high-frequency control, network latency and jitter can severely destabilize the system.<n>We propose Speculative Policy Orchestration (SPO), a latency-resilient cloud-edge framework.
  arXiv  Detail & Related papers  (2026-03-19T19:24:14Z)
• Scaling Attention via Feature Sparsity [50.64995497733461]
  Scaling Transformers to ultra-long contexts is bottlenecked by the $O(n2 d)$ cost of self-attention.<n>We propose Sparse Feature Attention, where queries and keys are represented as $k$-sparse codes that preserve high-dimensional expressivity.<n>Across GPT-2 and Qwen3 pretraining, SFA matches dense baselines while improving speed by up to $2.5times$ and reducing FLOPs and KV-cache by nearly 50%.
  arXiv  Detail & Related papers  (2026-03-17T08:41:50Z)
• Cloud-OpsBench: A Reproducible Benchmark for Agentic Root Cause Analysis in Cloud Systems [51.2882705779387]
  Cloud-OpsBench is a large-scale benchmark that employs a State Snapshot Paradigm to construct a deterministic digital twin of the cloud.<n>It features 452 distinct fault cases across 40 root cause types spanning the full stack.
  arXiv  Detail & Related papers  (2026-02-28T05:04:42Z)
• DropoutTS: Sample-Adaptive Dropout for Robust Time Series Forecasting [59.868414584142336]
  DropoutTS is a model-agnostic plugin that shifts the paradigm from "what" to "how much" to learn.<n>It maps noise to adaptive dropout rates - selectively suppressing spurious fluctuations while preserving fine-grained fidelity.
  arXiv  Detail & Related papers  (2026-01-29T13:49:20Z)
• ObCLIP: Oblivious CLoud-Device Hybrid Image Generation with Privacy Preservation [9.081441952478306]
  ObCLIP is a plug-and-play safeguard for oblivious cloud-device hybrid generation.<n>It provides rigorous privacy and comparable utility to cloud models with slightly increased server cost.
  arXiv  Detail & Related papers  (2025-10-05T11:09:10Z)
• Artificial Intelligence for Cost-Aware Resource Prediction in Big Data Pipelines [0.0]
  Over-provisioning leads to unnecessary costs, while under-provisioning risks performance degradation and SLA violations.<n>This work presents an artificial intelligence approach to predict resource utilization in big data pipelines using Random Forest regression.
  arXiv  Detail & Related papers  (2025-09-30T20:01:12Z)
• CloudFormer: An Attention-based Performance Prediction for Public Clouds with Unknown Workload [9.00677252346245]
  Cloud platforms are increasingly relied upon to host diverse, resource-intensive workloads.<n>Existing management techniques, including VM scheduling and resource provisioning, require accurate performance prediction.<n>We propose CloudFormer, a dual-branch Transformer-based model designed to predict VM performance degradation in black-box environments.
  arXiv  Detail & Related papers  (2025-09-03T15:15:44Z)
• Cost-Aware Contrastive Routing for LLMs [57.30288453580456]
  We introduce Cost-Spectrum Contrastive Routing (CSCR), a lightweight framework that maps both prompts and models into a shared embedding space.<n>CSCR consistently outperforms baselines, improving the accuracy-cost tradeoff by up to 25%.
  arXiv  Detail & Related papers  (2025-08-17T20:16:44Z)
• EC-Diff: Fast and High-Quality Edge-Cloud Collaborative Inference for Diffusion Models [57.059991285047296]
  hybrid edge-cloud collaborative framework was recently proposed to realize fast inference and high-quality generation.<n>Excessive cloud denoising prolongs inference time, while insufficient steps cause semantic ambiguity, leading to inconsistency in edge model output.<n>We propose EC-Diff that accelerates cloud inference through gradient-based noise estimation.<n>Our method significantly enhances generation quality compared to edge inference, while achieving up to an average $2times$ speedup in inference compared to cloud inference.
  arXiv  Detail & Related papers  (2025-07-16T07:23:14Z)
• PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning [54.99373314906667]
  Self-supervised representation learning for point cloud has demonstrated effectiveness in improving pre-trained model performance across diverse tasks.<n>As pre-trained models grow in complexity, fully fine-tuning them for downstream applications demands substantial computational and storage resources.<n>We propose PointLoRA, a simple yet effective method that combines low-rank adaptation (LoRA) with multi-scale token selection to efficiently fine-tune point cloud models.
  arXiv  Detail & Related papers  (2025-04-22T16:41:21Z)
• Device-Cloud Collaborative Recommendation via Meta Controller [65.97416287295152]
  We propose a meta controller to dynamically manage the collaboration between the on-device recommender and the cloud-based recommender. On the basis of the counterfactual samples and the extended training, extensive experiments in the industrial recommendation scenarios show the promise of meta controller.
  arXiv  Detail & Related papers  (2022-07-07T03:23:04Z)
• DeF-DReL: Systematic Deployment of Serverless Functions in Fog and Cloud environments using Deep Reinforcement Learning [8.204696165200577]
  Fog environment made its limited resource available to a large number of users to deploy their serverless applications. Recent research mainly focuses on assigning maximum resources to such applications from the fog node and not taking full advantage of the cloud environment. We propose DeF-DReL, a Systematic Deployment of Serverless Functions in Fog and Cloud environments using Deep Reinforcement Learning.
  arXiv  Detail & Related papers  (2021-10-29T12:10:54Z)
• A Predictive Autoscaler for Elastic Batch Jobs [8.354712625979776]
  Large batch jobs such as Deep Learning, HPC and Spark require far more computational resources and higher cost than conventional online service. We propose a predictive autoscaler to provide an elastic interface for the customers and overprovision instances.
  arXiv  Detail & Related papers  (2020-10-10T17:35:55Z)
• Budget Learning via Bracketing [50.085728094234476]
  The budget learning problem poses the learner's goal as minimising use of the cloud while suffering no discernible loss in accuracy. We propose a new formulation for the BL problem via the concept of bracketings. We empirically validate our theory on real-world datasets, demonstrating improved performance over prior gating based methods.
  arXiv  Detail & Related papers  (2020-04-14T04:38:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.