Reinforcement Learning-based Dynamic Service Placement in Vehicular Networks

• URL: http://arxiv.org/abs/2105.15022v2
• Date: Tue, 1 Jun 2021 13:38:15 GMT
• Title: Reinforcement Learning-based Dynamic Service Placement in Vehicular Networks
• Authors: Anum Talpur and Mohan Gurusamy
• Abstract summary: complexity of traffic mobility patterns and dynamics in the requests for different types of services has made service placement a challenging task. A typical static placement solution is not effective as it does not consider the traffic mobility and service dynamics. We propose a reinforcement learning-based dynamic (RL-Dynamic) service placement framework to find the optimal placement of services at the edge servers.
• Score: 4.010371060637208
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The emergence of technologies such as 5G and mobile edge computing has enabled provisioning of different types of services with different resource and service requirements to the vehicles in a vehicular network.The growing complexity of traffic mobility patterns and dynamics in the requests for different types of services has made service placement a challenging task. A typical static placement solution is not effective as it does not consider the traffic mobility and service dynamics. In this paper, we propose a reinforcement learning-based dynamic (RL-Dynamic) service placement framework to find the optimal placement of services at the edge servers while considering the vehicle's mobility and dynamics in the requests for different types of services. We use SUMO and MATLAB to carry out simulation experiments. In our learning framework, for the decision module, we consider two alternative objective functions-minimizing delay and minimizing edge server utilization. We developed an ILP based problem formulation for the two objective functions. The experimental results show that 1) compared to static service placement, RL-based dynamic service placement achieves fair utilization of edge server resources and low service delay, and 2) compared to delay-optimized placement, server utilization optimized placement utilizes resources more effectively, achieving higher fairness with lower edge-server utilization.

Related papers

• Resource Allocation for Twin Maintenance and Computing Task Processing in Digital Twin Vehicular Edge Computing Network [48.15151800771779]
  Vehicle edge computing (VEC) can provide computing caching services by deploying VEC servers near vehicles. However, VEC networks still face challenges such as high vehicle mobility. This study examines two types of delays caused by twin processing within the network.
  arXiv  Detail & Related papers  (2024-07-10T12:08:39Z)
• DNN Partitioning, Task Offloading, and Resource Allocation in Dynamic Vehicular Networks: A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approach [49.56404236394601]
  We formulate the problem of joint DNN partitioning, task offloading, and resource allocation in Vehicular Edge Computing. Our objective is to minimize the DNN-based task completion time while guaranteeing the system stability over time. We propose a Multi-Agent Diffusion-based Deep Reinforcement Learning (MAD2RL) algorithm, incorporating the innovative use of diffusion models.
  arXiv  Detail & Related papers  (2024-06-11T06:31:03Z)
• A Learning-based Incentive Mechanism for Mobile AIGC Service in Decentralized Internet of Vehicles [49.86094523878003]
  We propose a decentralized incentive mechanism for mobile AIGC service allocation. We employ multi-agent deep reinforcement learning to find the balance between the supply of AIGC services on RSUs and user demand for services within the IoV context.
  arXiv  Detail & Related papers  (2024-03-29T12:46:07Z)
• Edge computing service deployment and task offloading based on multi-task high-dimensional multi-objective optimization [5.64850919046892]
  This study investigates service deployment and task offloading challenges in a multi-user environment. To ensure stable service provisioning, beyond considering latency, energy consumption, and cost, network reliability is also incorporated. To promote equitable usage of edge servers, load balancing is introduced as a fourth task offloading objective.
  arXiv  Detail & Related papers  (2023-12-07T07:30:47Z)
• Real-time Control of Electric Autonomous Mobility-on-Demand Systems via Graph Reinforcement Learning [14.073588678179865]
  Electric Autonomous Mobility-on-Demand (E-AMoD) fleets need to make several real-time decisions. We present the E-AMoD control problem through the lens of reinforcement learning. We propose a graph network-based framework to achieve drastically improved scalability and superior performance overoptimals.
  arXiv  Detail & Related papers  (2023-11-09T22:57:21Z)
• Adaptive Resource Allocation for Virtualized Base Stations in O-RAN with Online Learning [60.17407932691429]
  Open Radio Access Network systems, with their base stations (vBSs), offer operators the benefits of increased flexibility, reduced costs, vendor diversity, and interoperability. We propose an online learning algorithm that balances the effective throughput and vBS energy consumption, even under unforeseeable and "challenging'' environments. We prove the proposed solutions achieve sub-linear regret, providing zero average optimality gap even in challenging environments.
  arXiv  Detail & Related papers  (2023-09-04T17:30:21Z)
• Dynamic Resource Allocation for Metaverse Applications with Deep Reinforcement Learning [64.75603723249837]
  This work proposes a novel framework to dynamically manage and allocate different types of resources for Metaverse applications. We first propose an effective solution to divide applications into groups, namely MetaInstances, where common functions can be shared among applications. Then, to capture the real-time, dynamic, and uncertain characteristics of request arrival and application departure processes, we develop a semi-Markov decision process-based framework.
  arXiv  Detail & Related papers  (2023-02-27T00:30:01Z)
• Knowledge Transfer in Deep Reinforcement Learning for Slice-Aware Mobility Robustness Optimization [0.8937905773981699]
  We propose a deep reinforcement learning-based slice-aware mobility robustness optimization (SAMRO) approach. It improves handover performance with per-slice service assurance by optimizing slice-specific handover parameters. We also develop a two-step transfer learning scheme: 1) regularized offline reinforcement learning, and 2) effective online fine-tuning with mixed experience replay.
  arXiv  Detail & Related papers  (2022-03-07T09:26:15Z)
• DRLD-SP: A Deep Reinforcement Learning-based Dynamic Service Placement in Edge-Enabled Internet of Vehicles [4.010371060637208]
  5G and edge computing has enabled the emergence of Internet of Vehicles (IoV) limited resources at the edge, high mobility of vehicles, increasing demand, and dynamicity in service request-types have made service placement a challenging task. A typical static placement solution is not effective as it does not consider the traffic mobility and service dynamics. We propose a Deep Reinforcement Learning-based Dynamic Service Placement framework with the objective of minimizing the maximum edge resource usage and service delay.
  arXiv  Detail & Related papers  (2021-06-11T10:17:27Z)
• Multi-Exit Semantic Segmentation Networks [78.44441236864057]
  We propose a framework for converting state-of-the-art segmentation models to MESS networks. specially trained CNNs that employ parametrised early exits along their depth to save during inference on easier samples. We co-optimise the number, placement and architecture of the attached segmentation heads, along with the exit policy, to adapt to the device capabilities and application-specific requirements.
  arXiv  Detail & Related papers  (2021-06-07T11:37:03Z)
• AI-based Resource Allocation: Reinforcement Learning for Adaptive Auto-scaling in Serverless Environments [0.0]
  Serverless computing has emerged as a compelling new paradigm of cloud computing models in recent years. A common approach among both commercial and open source serverless computing platforms is workload-based auto-scaling. In this paper we investigate the applicability of a reinforcement learning approach to request-based auto-scaling in a serverless framework.
  arXiv  Detail & Related papers  (2020-05-29T06:18:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.