A Deep RL Approach on Task Placement and Scaling of Edge Resources for Cellular Vehicle-to-Network Service Provisioning

• URL: http://arxiv.org/abs/2305.09832v2
• Date: Tue, 9 Jul 2024 16:42:37 GMT
• Title: A Deep RL Approach on Task Placement and Scaling of Edge Resources for Cellular Vehicle-to-Network Service Provisioning
• Authors: Cyril Shih-Huan Hsu, Jorge Martín-Pérez, Danny De Vleeschauwer, Koteswararao Kondepu, Luca Valcarenghi, Xi Li, Chrysa Papagianni,
• Abstract summary: We tackle the interdependent problems of service task placement and scaling of edge resources. We introduce a Deep Hybrid Policy Gradient (DHPG), a Deep Reinforcement Learning (DRL) approach for hybrid action spaces. The performance of DHPG is evaluated against several state-of-the-art (SoA) solutions through simulations employing a real-world C-V2N traffic dataset.
• Score: 6.625994697789603
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cellular-Vehicle-to-Everything (C-V2X) is currently at the forefront of the digital transformation of our society. By enabling vehicles to communicate with each other and with the traffic environment using cellular networks, we redefine transportation, improving road safety and transportation services, increasing efficiency of vehicular traffic flows, and reducing environmental impact. To effectively facilitate the provisioning of Cellular Vehicular-to-Network (C-V2N) services, we tackle the interdependent problems of service task placement and scaling of edge resources. Specifically, we formulate the joint problem and prove that it is not computationally tractable. To address its complexity we introduce a Deep Hybrid Policy Gradient (DHPG), a Deep Reinforcement Learning (DRL) approach for hybrid action spaces.The performance of DHPG is evaluated against several state-of-the-art (SoA) solutions through simulations employing a real-world C-V2N traffic dataset. The results demonstrate that DHPG outperforms SoA solutions in maintaining C-V2N service latency below the preset delay threshold, while simultaneously optimizing the utilization of computing resources. Finally, time complexity analysis is conducted to verify that the proposed approach can support real-time C-V2N services.

Related papers

• Communication-Aware Consistent Edge Selection for Mobile Users and Autonomous Vehicles [1.2453219864236245]
  Offloading time-sensitive, computationally intensive tasks can enhance service efficiency. This paper proposes a deep reinforcement learning framework based on the Deep Deterministic Policy Gradient (DDPG) algorithm. A joint allocation method of communication and switching of APs is proposed to minimize computational load, service latency, and interruptions.
  arXiv  Detail & Related papers  (2024-08-06T20:21:53Z)
• Resource Allocation for Twin Maintenance and Computing Task Processing in Digital Twin Vehicular Edge Computing Network [48.15151800771779]
  Vehicle edge computing (VEC) can provide computing caching services by deploying VEC servers near vehicles. However, VEC networks still face challenges such as high vehicle mobility. This study examines two types of delays caused by twin processing within the network.
  arXiv  Detail & Related papers  (2024-07-10T12:08:39Z)
• Deep-Reinforcement-Learning-Based AoI-Aware Resource Allocation for RIS-Aided IoV Networks [43.443526528832145]
  We propose a RIS-assisted internet of vehicles (IoV) network, considering the vehicle-to-everything (V2X) communication method. In order to improve the timeliness of vehicle-to-infrastructure (V2I) links and the stability of vehicle-to-vehicle (V2V) links, we introduce the age of information (AoI) model and the payload transmission probability model.
  arXiv  Detail & Related papers  (2024-06-17T06:16:07Z)
• DNN Partitioning, Task Offloading, and Resource Allocation in Dynamic Vehicular Networks: A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approach [49.56404236394601]
  We formulate the problem of joint DNN partitioning, task offloading, and resource allocation in Vehicular Edge Computing. Our objective is to minimize the DNN-based task completion time while guaranteeing the system stability over time. We propose a Multi-Agent Diffusion-based Deep Reinforcement Learning (MAD2RL) algorithm, incorporating the innovative use of diffusion models.
  arXiv  Detail & Related papers  (2024-06-11T06:31:03Z)
• Scheduling Out-of-Coverage Vehicular Communications Using Reinforcement Learning [3.058685580689605]
  We propose VRLS (Vehicular Reinforcement Learning Scheduler), a centralized scheduler that proactively assigns resources for out-of-coverage V2V communications. We evaluate the performance of VRLS under varying mobility, network load, wireless channel, and resource configurations.
  arXiv  Detail & Related papers  (2022-07-13T22:47:48Z)
• AI-aided Traffic Control Scheme for M2M Communications in the Internet of Vehicles [61.21359293642559]
  The dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies. We consider a hybrid traffic control scheme and use proximal policy optimization (PPO) method to tackle it.
  arXiv  Detail & Related papers  (2022-03-05T10:54:05Z)
• An Energy Consumption Model for Electrical Vehicle Networks via Extended Federated-learning [50.85048976506701]
  This paper proposes a novel solution to range anxiety based on a federated-learning model. It is capable of estimating battery consumption and providing energy-efficient route planning for vehicle networks.
  arXiv  Detail & Related papers  (2021-11-13T15:03:44Z)
• Computation Offloading in Heterogeneous Vehicular Edge Networks: On-line and Off-policy Bandit Solutions [30.606518785629046]
  In a fast-varying vehicular environment, the latency in offloading arises as a result of network congestion. We propose an on-line algorithm and an off-policy learning algorithm based on bandit theory. We show that the proposed solutions adapt to the traffic changes of the network by selecting the least congested network.
  arXiv  Detail & Related papers  (2020-08-14T11:48:13Z)
• Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach [88.45509934702913]
  We design a navigation policy for multiple unmanned aerial vehicles (UAVs) where mobile base stations (BSs) are deployed. We incorporate different contextual information such as energy and age of information (AoI) constraints to ensure the data freshness at the ground BS. By applying the proposed trained model, an effective real-time trajectory policy for the UAV-BSs captures the observable network states over time.
  arXiv  Detail & Related papers  (2020-02-21T07:29:15Z)
• Reinforcement Learning Based Vehicle-cell Association Algorithm for Highly Mobile Millimeter Wave Communication [53.47785498477648]
  This paper investigates the problem of vehicle-cell association in millimeter wave (mmWave) communication networks. We first formulate the user state (VU) problem as a discrete non-vehicle association optimization problem. The proposed solution achieves up to 15% gains in terms sum of user complexity and 20% reduction in VUE compared to several baseline designs.
  arXiv  Detail & Related papers  (2020-01-22T08:51:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.