Reinforcement Learning (RL) Augmented Cold Start Frequency Reduction in
Serverless Computing
- URL: http://arxiv.org/abs/2308.07541v1
- Date: Tue, 15 Aug 2023 03:01:41 GMT
- Title: Reinforcement Learning (RL) Augmented Cold Start Frequency Reduction in
Serverless Computing
- Authors: Siddharth Agarwal, Maria A. Rodriguez, Rajkumar Buyya
- Abstract summary: This work focuses on reducing the frequency of cold starts on the Function-as-a-Service platform by using Reinforcement Learning.
The proposed solution was implemented on Kubeless and was evaluated using a normalised real-world function demand trace with matrix multiplication as the workload.
- Score: 20.638612359627952
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Function-as-a-Service is a cloud computing paradigm offering an event-driven
execution model to applications. It features serverless attributes by
eliminating resource management responsibilities from developers and offers
transparent and on-demand scalability of applications. Typical serverless
applications have stringent response time and scalability requirements and
therefore rely on deployed services to provide quick and fault-tolerant
feedback to clients. However, the FaaS paradigm suffers from cold starts as
there is a non-negligible delay associated with on-demand function
initialization. This work focuses on reducing the frequency of cold starts on
the platform by using Reinforcement Learning. Our approach uses Q-learning and
considers metrics such as function CPU utilization, existing function
instances, and response failure rate to proactively initialize functions in
advance based on the expected demand. The proposed solution was implemented on
Kubeless and was evaluated using a normalised real-world function demand trace
with matrix multiplication as the workload. The results demonstrate a
favourable performance of the RL-based agent when compared to Kubeless' default
policy and function keep-alive policy by improving throughput by up to 8.81%
and reducing computation load and resource wastage by up to 55% and 37%,
respectively, which is a direct outcome of reduced cold starts.
Related papers
- SPES: Towards Optimizing Performance-Resource Trade-Off for Serverless Functions [31.01399126339857]
Serverless computing is gaining traction due to its efficiency and ability to harness on-demand cloud resources.
Existing solutions tend to use over-simplistic strategies for function pre-loading/unloading without full invocation pattern exploitation.
We propose SPES, the first differentiated scheduler for runtime cold start mitigation by optimizing serverless function provision.
arXiv Detail & Related papers (2024-03-26T10:28:41Z) - Shabari: Delayed Decision-Making for Faster and Efficient Serverless
Functions [0.30693357740321775]
We introduce Shabari, a resource management framework for serverless systems.
Shabari makes decisions as late as possible to right-size each invocation to meet functions' performance objectives.
For a range of serverless functions and inputs, Shabari reduces SLO violations by 11-73%.
arXiv Detail & Related papers (2024-01-16T22:20:36Z) - Augmenting Unsupervised Reinforcement Learning with Self-Reference [63.68018737038331]
Humans possess the ability to draw on past experiences explicitly when learning new tasks.
We propose the Self-Reference (SR) approach, an add-on module explicitly designed to leverage historical information.
Our approach achieves state-of-the-art results in terms of Interquartile Mean (IQM) performance and Optimality Gap reduction on the Unsupervised Reinforcement Learning Benchmark.
arXiv Detail & Related papers (2023-11-16T09:07:34Z) - Resource-Aware Hierarchical Federated Learning for Video Caching in
Wireless Networks [29.137803674759848]
A privacy-preserving method is desirable to learn how users' demands change over time.
This paper proposes a novel resource-aware hierarchical federated learning (RawHFL) solution to predict users' future content requests.
Our simulation results show that the proposed solution significantly outperforms the considered baselines in terms of prediction accuracy and total energy expenditure.
arXiv Detail & Related papers (2023-11-12T18:23:17Z) - Client Orchestration and Cost-Efficient Joint Optimization for
NOMA-Enabled Hierarchical Federated Learning [55.49099125128281]
We propose a non-orthogonal multiple access (NOMA) enabled HFL system under semi-synchronous cloud model aggregation.
We show that the proposed scheme outperforms the considered benchmarks regarding HFL performance improvement and total cost reduction.
arXiv Detail & Related papers (2023-11-03T13:34:44Z) - A Deep Recurrent-Reinforcement Learning Method for Intelligent
AutoScaling of Serverless Functions [21.260954070091]
We investigate a model-free Recurrent RL agent for function autoscaling and compare it against the model-free Proximal Policy optimisation algorithm.
We find that a LSTM-based autoscaling agent is able to improve throughput by 18%, function execution by 13% and account for 8.4% more function instances.
arXiv Detail & Related papers (2023-08-11T04:41:19Z) - Differentially Private Deep Q-Learning for Pattern Privacy Preservation
in MEC Offloading [76.0572817182483]
attackers may eavesdrop on the offloading decisions to infer the edge server's (ES's) queue information and users' usage patterns.
We propose an offloading strategy which jointly minimizes the latency, ES's energy consumption, and task dropping rate, while preserving pattern privacy (PP)
We develop a Differential Privacy Deep Q-learning based Offloading (DP-DQO) algorithm to solve this problem while addressing the PP issue by injecting noise into the generated offloading decisions.
arXiv Detail & Related papers (2023-02-09T12:50:18Z) - FIRE: A Failure-Adaptive Reinforcement Learning Framework for Edge
Computing Migrations [55.131858975133085]
FIRE is a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment.
We propose ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function.
We show that FIRE reduces costs compared to vanilla RL and the greedy baseline in the event of failures.
arXiv Detail & Related papers (2022-09-28T19:49:39Z) - Harvesting Idle Resources in Serverless Computing via Reinforcement
Learning [7.346628578439277]
FRM maximizes resource efficiency by dynamically harvesting idle resources from functions over-supplied to functions under-supplied.
FRM monitors each function's resource utilization in real-time, detects over-provisioning and under-provisioning, and applies deep reinforcement learning to harvest idle resources safely.
We have implemented and deployed a FRM prototype in a 13-node Apache OpenWhisk cluster.
arXiv Detail & Related papers (2021-08-28T23:02:56Z) - Joint Client Scheduling and Resource Allocation under Channel
Uncertainty in Federated Learning [47.97586668316476]
Federated learning (FL) over wireless networks depends on the reliability of the client-server connectivity and clients' local computation capabilities.
In this article, we investigate the problem of client scheduling and resource block (RB) allocation to enhance the performance of model training using FL.
A proposed method reduces the gap of the training accuracy loss by up to 40.7% compared to state-of-theart client scheduling and RB allocation methods.
arXiv Detail & Related papers (2021-06-12T15:18:48Z) - Multi-Armed Bandit Based Client Scheduling for Federated Learning [91.91224642616882]
federated learning (FL) features ubiquitous properties such as reduction of communication overhead and preserving data privacy.
In each communication round of FL, the clients update local models based on their own data and upload their local updates via wireless channels.
This work provides a multi-armed bandit-based framework for online client scheduling (CS) in FL without knowing wireless channel state information and statistical characteristics of clients.
arXiv Detail & Related papers (2020-07-05T12:32:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.