Double Deep Q-Learning-based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications

• URL: http://arxiv.org/abs/2309.10180v1
• Date: Mon, 18 Sep 2023 22:17:23 GMT
• Title: Double Deep Q-Learning-based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications
• Authors: Masoud Shokrnezhad, Tarik Taleb, and Patrizio Dazzi
• Abstract summary: This paper studies the joint problem of communication and computing resource allocation, dubbed CCRA, to minimize total cost. We formulate the problem as a non-linear programming model and propose two approaches, dubbed B&B-CCRA and WF-CCRA, based on the Branch & Bound and Water-Filling algorithms. Numerical simulations show that B&B-CCRA optimally solves the problem, whereas WF-CCRA delivers near-optimal solutions in a substantially shorter time.
• Score: 11.864695986880347
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Nowadays, as the need for capacity continues to grow, entirely novel services are emerging. A solid cloud-network integrated infrastructure is necessary to supply these services in a real-time responsive, and scalable way. Due to their diverse characteristics and limited capacity, communication and computing resources must be collaboratively managed to unleash their full potential. Although several innovative methods have been proposed to orchestrate the resources, most ignored network resources or relaxed the network as a simple graph, focusing only on cloud resources. This paper fills the gap by studying the joint problem of communication and computing resource allocation, dubbed CCRA, including function placement and assignment, traffic prioritization, and path selection considering capacity constraints and quality requirements, to minimize total cost. We formulate the problem as a non-linear programming model and propose two approaches, dubbed B\&B-CCRA and WF-CCRA, based on the Branch \& Bound and Water-Filling algorithms to solve it when the system is fully known. Then, for partially known systems, a Double Deep Q-Learning (DDQL) architecture is designed. Numerical simulations show that B\&B-CCRA optimally solves the problem, whereas WF-CCRA delivers near-optimal solutions in a substantially shorter time. Furthermore, it is demonstrated that DDQL-CCRA obtains near-optimal solutions in the absence of request-specific information.

Related papers

• Two-Timescale Model Caching and Resource Allocation for Edge-Enabled AI-Generated Content Services [55.0337199834612]
  Generative AI (GenAI) has emerged as a transformative technology, enabling customized and personalized AI-generated content (AIGC) services. These services require executing GenAI models with billions of parameters, posing significant obstacles to resource-limited wireless edge. We introduce the formulation of joint model caching and resource allocation for AIGC services to balance a trade-off between AIGC quality and latency metrics.
  arXiv  Detail & Related papers  (2024-11-03T07:01:13Z)
• Joint Admission Control and Resource Allocation of Virtual Network Embedding via Hierarchical Deep Reinforcement Learning [69.00997996453842]
  We propose a deep Reinforcement Learning approach to learn a joint Admission Control and Resource Allocation policy for virtual network embedding. We show that HRL-ACRA outperforms state-of-the-art baselines in terms of both the acceptance ratio and long-term average revenue.
  arXiv  Detail & Related papers  (2024-06-25T07:42:30Z)
• ORIENT: A Priority-Aware Energy-Efficient Approach for Latency-Sensitive Applications in 6G [15.753216159980434]
  Growing concerns about increased energy consumption in computing and networking. The expected surge in connected devices and resource-demanding applications presents unprecedented challenges for energy resources. We investigate the joint problem of service instance placement and assignment, path selection, and request prioritization, dubbed PIRA.
  arXiv  Detail & Related papers  (2024-02-10T12:05:52Z)
• Elastic Entangled Pair and Qubit Resource Management in Quantum Cloud Computing [73.7522199491117]
  Quantum cloud computing (QCC) offers a promising approach to efficiently provide quantum computing resources. The fluctuations in user demand and quantum circuit requirements are challenging for efficient resource provisioning. We propose a resource allocation model to provision quantum computing and networking resources.
  arXiv  Detail & Related papers  (2023-07-25T00:38:46Z)
• Joint Service Caching, Communication and Computing Resource Allocation in Collaborative MEC Systems: A DRL-based Two-timescale Approach [15.16859210403316]
  Meeting the strict Quality of Service (QoS) requirements of terminals has imposed a challenge on Multiaccess Edge Computing (MEC) systems. We propose a collaborative framework that facilitates resource sharing between the edge servers. We show that our proposed algorithm outperforms the baseline algorithms in terms of the average switching and cache cost.
  arXiv  Detail & Related papers  (2023-07-19T00:27:49Z)
• Multi-Resource Allocation for On-Device Distributed Federated Learning Systems [79.02994855744848]
  This work poses a distributed multi-resource allocation scheme for minimizing the weighted sum of latency and energy consumption in the on-device distributed federated learning (FL) system. Each mobile device in the system engages the model training process within the specified area and allocates its computation and communication resources for deriving and uploading parameters, respectively.
  arXiv  Detail & Related papers  (2022-11-01T14:16:05Z)
• Network Aware Compute and Memory Allocation in Optically Composable Data Centres with Deep Reinforcement Learning and Graph Neural Networks [0.0]
  Resource-disaggregated data centre architectures promise a means of pooling resources remotely within data centres. We show how this can be done using an optically switched circuit circuit backbone in the data centre network (DCN) We show how emphdeep reinforcement learning can be used to learn effective emphnetwork-aware and emphtopologically-scalable allocation policies end-to-end.
  arXiv  Detail & Related papers  (2022-10-26T09:46:50Z)
• Artificial Intelligence Empowered Multiple Access for Ultra Reliable and Low Latency THz Wireless Networks [76.89730672544216]
  Terahertz (THz) wireless networks are expected to catalyze the beyond fifth generation (B5G) era. To satisfy the ultra-reliability and low-latency demands of several B5G applications, novel mobility management approaches are required. This article presents a holistic MAC layer approach that enables intelligent user association and resource allocation, as well as flexible and adaptive mobility management.
  arXiv  Detail & Related papers  (2022-08-17T03:00:24Z)
• CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing [19.990451009223573]
  Network slicing is proposed as a promising solution for resource utilization in 5G and future networks. We formulate the problem as a Constrained Markov Decision Process (CMDP) without knowing models and hidden structures. We propose to solve the problem using CLARA, a Constrained reinforcement LeArning based Resource Allocation algorithm.
  arXiv  Detail & Related papers  (2021-11-16T11:54:09Z)
• Resource Allocation via Model-Free Deep Learning in Free Space Optical Communications [119.81868223344173]
  The paper investigates the general problem of resource allocation for mitigating channel fading effects in Free Space Optical (FSO) communications. Under this framework, we propose two algorithms that solve FSO resource allocation problems.
  arXiv  Detail & Related papers  (2020-07-27T17:38:51Z)
• A Review on Computational Intelligence Techniques in Cloud and Edge Computing [14.610305602165415]
  Cloud computing (CC) is a centralized computing paradigm that accumulates resources provides these resources to users through Internet. Centrally distributed computing (EC) provides resources in a decentralized manner, which can respond to users' needs faster. As both CC and EC are resource-sensitive, several big issues arise, such as how to conduct job scheduling, resource allocation, and task offloading.
  arXiv  Detail & Related papers  (2020-07-27T09:29:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.