Job Scheduling in Datacenters using Constraint Controlled RL
- URL: http://arxiv.org/abs/2211.05338v1
- Date: Thu, 10 Nov 2022 04:43:14 GMT
- Title: Job Scheduling in Datacenters using Constraint Controlled RL
- Authors: Vanamala Venkataswamy
- Abstract summary: We apply Proportional-Integral-Derivative (PID) Lagrangian methods in Deep Reinforcement Learning to job scheduling problem in the green datacenter environment.
Experiments demonstrate improved performance compared to scheduling policies without the PID Lagrangian methods.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: This paper studies a model for online job scheduling in green datacenters. In
green datacenters, resource availability depends on the power supply from the
renewables. Intermittent power supply from renewables leads to intermittent
resource availability, inducing job delays (and associated costs). Green
datacenter operators must intelligently manage their workloads and available
power supply to extract maximum benefits. The scheduler's objective is to
schedule jobs on a set of resources to maximize the total value (revenue) while
minimizing the overall job delay. A trade-off exists between achieving high job
value on the one hand and low expected delays on the other. Hence, the aims of
achieving high rewards and low costs are in opposition. In addition, datacenter
operators often prioritize multiple objectives, including high system
utilization and job completion. To accomplish the opposing goals of maximizing
total job value and minimizing job delays, we apply the
Proportional-Integral-Derivative (PID) Lagrangian methods in Deep Reinforcement
Learning to job scheduling problem in the green datacenter environment.
Lagrangian methods are widely used algorithms for constrained optimization
problems. We adopt a controls perspective to learn the Lagrange multiplier with
proportional, integral, and derivative control, achieving favorable learning
dynamics. Feedback control defines cost terms for the learning agent, monitors
the cost limits during training, and continuously adjusts the learning
parameters to achieve stable performance. Our experiments demonstrate improved
performance compared to scheduling policies without the PID Lagrangian methods.
Experimental results illustrate the effectiveness of the Constraint Controlled
Reinforcement Learning (CoCoRL) scheduler that simultaneously satisfies
multiple objectives.
Related papers
- Goal-Conditioned Terminal Value Estimation for Real-time and Multi-task Model Predictive Control [1.2687745030755995]
We develop an MPC framework with goal-conditioned terminal value learning to achieve multitask policy optimization.
We evaluate the proposed method on a bipedal inverted pendulum robot model and confirm that combining goal-conditioned terminal value learning with an upper-level trajectory planner enables real-time control.
arXiv Detail & Related papers (2024-10-07T11:19:23Z) - Load Balancing in Federated Learning [3.2999744336237384]
Federated Learning (FL) is a decentralized machine learning framework that enables learning from data distributed across multiple remote devices.
This paper proposes a load metric for scheduling policies based on the Age of Information.
We establish the optimal parameters of the Markov chain model and validate our approach through simulations.
arXiv Detail & Related papers (2024-08-01T00:56:36Z) - Characterization of Large Language Model Development in the Datacenter [55.9909258342639]
Large Language Models (LLMs) have presented impressive performance across several transformative tasks.
However, it is non-trivial to efficiently utilize large-scale cluster resources to develop LLMs.
We present an in-depth characterization study of a six-month LLM development workload trace collected from our GPU datacenter Acme.
arXiv Detail & Related papers (2024-03-12T13:31:14Z) - CUDC: A Curiosity-Driven Unsupervised Data Collection Method with
Adaptive Temporal Distances for Offline Reinforcement Learning [62.58375643251612]
We propose a Curiosity-driven Unsupervised Data Collection (CUDC) method to expand feature space using adaptive temporal distances for task-agnostic data collection.
With this adaptive reachability mechanism in place, the feature representation can be diversified, and the agent can navigate itself to collect higher-quality data with curiosity.
Empirically, CUDC surpasses existing unsupervised methods in efficiency and learning performance in various downstream offline RL tasks of the DeepMind control suite.
arXiv Detail & Related papers (2023-12-19T14:26:23Z) - Dynamic Scheduling for Federated Edge Learning with Streaming Data [56.91063444859008]
We consider a Federated Edge Learning (FEEL) system where training data are randomly generated over time at a set of distributed edge devices with long-term energy constraints.
Due to limited communication resources and latency requirements, only a subset of devices is scheduled for participating in the local training process in every iteration.
arXiv Detail & Related papers (2023-05-02T07:41:16Z) - Sustainable AIGC Workload Scheduling of Geo-Distributed Data Centers: A
Multi-Agent Reinforcement Learning Approach [48.18355658448509]
Recent breakthroughs in generative artificial intelligence have triggered a surge in demand for machine learning training, which poses significant cost burdens and environmental challenges due to its substantial energy consumption.
Scheduling training jobs among geographically distributed cloud data centers unveils the opportunity to optimize the usage of computing capacity powered by inexpensive and low-carbon energy.
We propose an algorithm based on multi-agent reinforcement learning and actor-critic methods to learn the optimal collaborative scheduling strategy through interacting with a cloud system built with real-life workload patterns, energy prices, and carbon intensities.
arXiv Detail & Related papers (2023-04-17T02:12:30Z) - RARE: Renewable Energy Aware Resource Management in Datacenters [9.488752723308954]
Hyperscale cloud providers have announced plans to power their datacenters using renewable energy.
Integrating renewables to power the datacenters is challenging because the power generation is intermittent.
We present a scheduler that learns effective job scheduling policies while continually adapting to the intermittent power supply from renewables.
arXiv Detail & Related papers (2022-11-10T05:17:14Z) - Optimizing Data Collection for Machine Learning [87.37252958806856]
Modern deep learning systems require huge data sets to achieve impressive performance.
Over-collecting data incurs unnecessary present costs, while under-collecting may incur future costs and delay.
We propose a new paradigm for modeling the data collection as a formal optimal data collection problem.
arXiv Detail & Related papers (2022-10-03T21:19:05Z) - How Much More Data Do I Need? Estimating Requirements for Downstream
Tasks [99.44608160188905]
Given a small training data set and a learning algorithm, how much more data is necessary to reach a target validation or test performance?
Overestimating or underestimating data requirements incurs substantial costs that could be avoided with an adequate budget.
Using our guidelines, practitioners can accurately estimate data requirements of machine learning systems to gain savings in both development time and data acquisition costs.
arXiv Detail & Related papers (2022-07-04T21:16:05Z) - ANDREAS: Artificial intelligence traiNing scheDuler foR accElerAted
resource clusterS [1.798617052102518]
We propose ANDREAS, an advanced scheduling solution to maximize performance and minimize Data Centers operational costs.
experiments show that we can achieve a cost reduction between 30 and 62% on average with respect to first-principle methods.
arXiv Detail & Related papers (2021-05-11T14:36:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.