Reinforcement Learning-based Admission Control in Delay-sensitive Service Systems

• URL: http://arxiv.org/abs/2008.09590v1
• Date: Fri, 21 Aug 2020 17:33:55 GMT
• Title: Reinforcement Learning-based Admission Control in Delay-sensitive Service Systems
• Authors: Majid Raeis, Ali Tizghadam and Alberto Leon-Garcia
• Abstract summary: We propose a reinforcement learning-based admission controller that guarantees a probabilistic upper-bound on the end-to-end delay of the service system. Our controller uses the queue length information of the network and requires no knowledge about the network topology or system parameters.
• Score: 10.089520556398574
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensuring quality of service (QoS) guarantees in service systems is a challenging task, particularly when the system is composed of more fine-grained services, such as service function chains. An important QoS metric in service systems is the end-to-end delay, which becomes even more important in delay-sensitive applications, where the jobs must be completed within a time deadline. Admission control is one way of providing end-to-end delay guarantee, where the controller accepts a job only if it has a high probability of meeting the deadline. In this paper, we propose a reinforcement learning-based admission controller that guarantees a probabilistic upper-bound on the end-to-end delay of the service system, while minimizes the probability of unnecessary rejections. Our controller only uses the queue length information of the network and requires no knowledge about the network topology or system parameters. Since long-term performance metrics are of great importance in service systems, we take an average-reward reinforcement learning approach, which is well suited to infinite horizon problems. Our evaluations verify that the proposed RL-based admission controller is capable of providing probabilistic bounds on the end-to-end delay of the network, without using system model information.

Related papers

• ReinWiFi: A Reinforcement-Learning-Based Framework for the Application-Layer QoS Optimization of WiFi Networks [6.566362478263619]
  In this paper, a reinforcement-learning-based scheduling framework is proposed to optimize the application-layer quality-of-service of a wireless local area network (WLAN) suffering from unknown interference. A novel Q-network is trained to map from the historical scheduling parameters and observations to the current scheduling action.
  arXiv  Detail & Related papers  (2024-05-06T14:44:06Z)
• Secure Deep Reinforcement Learning for Dynamic Resource Allocation in Wireless MEC Networks [46.689212344009015]
  This paper proposes a blockchain-secured deep reinforcement learning (BC-DRL) optimization framework for data management and resource allocation in mobile edge computing networks. We design a low-latency reputation-based proof-of-stake (RPoS) consensus protocol to select highly reliable blockchain-enabled BSs. We provide extensive simulation results and analysis to validate that our BC-DRL framework achieves higher security, reliability, and resource utilization efficiency than benchmark blockchain consensus protocols and MEC resource allocation algorithms.
  arXiv  Detail & Related papers  (2023-12-13T09:39:32Z)
• Guaranteed Dynamic Scheduling of Ultra-Reliable Low-Latency Traffic via Conformal Prediction [72.59079526765487]
  The dynamic scheduling of ultra-reliable and low-latency traffic (URLLC) in the uplink can significantly enhance the efficiency of coexisting services. The main challenge is posed by the uncertainty in the process of URLLC packet generation. We introduce a novel scheduler for URLLC packets that provides formal guarantees on reliability and latency irrespective of the quality of the URLLC traffic predictor.
  arXiv  Detail & Related papers  (2023-02-15T14:09:55Z)
• Improving the Performance of Robust Control through Event-Triggered Learning [74.57758188038375]
  We propose an event-triggered learning algorithm that decides when to learn in the face of uncertainty in the LQR problem. We demonstrate improved performance over a robust controller baseline in a numerical example.
  arXiv  Detail & Related papers  (2022-07-28T17:36:37Z)
• An Intelligent Deterministic Scheduling Method for Ultra-Low Latency Communication in Edge Enabled Industrial Internet of Things [19.277349546331557]
  Time Sensitive Network (TSN) is recently researched to realize low latency communication via deterministic scheduling. Non-collision theory based deterministic scheduling (NDS) method is proposed to achieve ultra-low latency communication for the time-sensitive flows. Experiment results demonstrate that NDS/DQS can well support deterministic ultra-low latency services and guarantee efficient bandwidth utilization.
  arXiv  Detail & Related papers  (2022-07-17T16:52:51Z)
• Networked Online Learning for Control of Safety-Critical Resource-Constrained Systems based on Gaussian Processes [9.544146562919792]
  We propose a novel networked online learning approach based on Gaussian process regression. We propose an effective data transmission scheme between the local system and the cloud taking bandwidth limitations and time delay of the transmission channel into account.
  arXiv  Detail & Related papers  (2022-02-23T13:12:12Z)
• Learning a Discrete Set of Optimal Allocation Rules in a Queueing System with Unknown Service Rate [1.4094389874355762]
  We study admission control for a system with unknown arrival and service rates. In our model, at every job arrival, a dispatcher decides to assign the job to an available server or block it. Our goal is to design a dispatching policy that maximizes the long-term average reward for the dispatcher.
  arXiv  Detail & Related papers  (2022-02-04T22:39:03Z)
• Federated Learning with Correlated Data: Taming the Tail for Age-Optimal Industrial IoT [55.62157530259969]
  We study a sensor's transmit power minimization subject to the peak-AoI requirement and a probabilistic constraint on queuing latency. We propose a local-model selection approach which accounts for correlation among the sensor's training data. Numerical results show the tradeoff between the transmit power, peak AoI, and delay's tail distribution.
  arXiv  Detail & Related papers  (2021-08-17T08:38:31Z)
• Delay-Tolerant Constrained OCO with Application to Network Resource Allocation [44.67787270821051]
  We consider online convex optimization (OCO) with multi-slot feedback delay. An agent makes a sequence of online decisions to minimize the accumulation of time-varying convex loss functions. We propose Delay-Tolerant Constrained-OCO, which uses a novel constraint penalty with double regularization to tackle the asynchrony between information feedback and decision updates.
  arXiv  Detail & Related papers  (2021-05-09T19:32:33Z)
• Better than the Best: Gradient-based Improper Reinforcement Learning for Network Scheduling [60.48359567964899]
  We consider the problem of scheduling in constrained queueing networks with a view to minimizing packet delay. We use a policy gradient based reinforcement learning algorithm that produces a scheduler that performs better than the available atomic policies.
  arXiv  Detail & Related papers  (2021-05-01T10:18:34Z)
• Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Service [1.8477401359673706]
  Servicerate control is a common mechanism for providing guarantees in service systems. In this paper, we introduce a reinforcement learning-based (RL-based) service-rate controller. Our controller provides explicit probabilistic guarantees on the end-to-end delay of the system.
  arXiv  Detail & Related papers  (2021-01-12T17:28:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.