Attention-based Open RAN Slice Management using Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2306.09490v1
• Date: Thu, 15 Jun 2023 20:37:19 GMT
• Title: Attention-based Open RAN Slice Management using Deep Reinforcement Learning
• Authors: Fatemeh Lotfi, Fatemeh Afghah, Jonathan Ashdown
• Abstract summary: This paper introduces an innovative attention-based deep RL (ADRL) technique that leverages the O-RAN disaggregated modules and distributed agent cooperation. Simulation results demonstrate significant improvements in network performance compared to other DRL baseline methods.
• Score: 6.177038245239758
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As emerging networks such as Open Radio Access Networks (O-RAN) and 5G continue to grow, the demand for various services with different requirements is increasing. Network slicing has emerged as a potential solution to address the different service requirements. However, managing network slices while maintaining quality of services (QoS) in dynamic environments is a challenging task. Utilizing machine learning (ML) approaches for optimal control of dynamic networks can enhance network performance by preventing Service Level Agreement (SLA) violations. This is critical for dependable decision-making and satisfying the needs of emerging networks. Although RL-based control methods are effective for real-time monitoring and controlling network QoS, generalization is necessary to improve decision-making reliability. This paper introduces an innovative attention-based deep RL (ADRL) technique that leverages the O-RAN disaggregated modules and distributed agent cooperation to achieve better performance through effective information extraction and implementing generalization. The proposed method introduces a value-attention network between distributed agents to enable reliable and optimal decision-making. Simulation results demonstrate significant improvements in network performance compared to other DRL baseline methods.

Related papers

• DRL Optimization Trajectory Generation via Wireless Network Intent-Guided Diffusion Models for Optimizing Resource Allocation [58.62766376631344]
  We propose a customized wireless network intent (WNI-G) model to address different state variations of wireless communication networks. Extensive simulation achieves greater stability in spectral efficiency and variations of traditional DRL models in dynamic communication systems.
  arXiv  Detail & Related papers  (2024-10-18T14:04:38Z)
• Meta Reinforcement Learning Approach for Adaptive Resource Optimization in O-RAN [6.326120268549892]
  Open Radio Access Network (O-RAN) addresses the variable demands of modern networks with unprecedented efficiency and adaptability. This paper proposes a novel Meta Deep Reinforcement Learning (Meta-DRL) strategy, inspired by Model-Agnostic Meta-Learning (MAML) to advance resource block and downlink power allocation in O-RAN.
  arXiv  Detail & Related papers  (2024-09-30T23:04:30Z)
• Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach [51.63921041249406]
  Non-orthogonal multiple access (NOMA) enables multiple users to share the same frequency band, and simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) deploying STAR-RIS indoors presents challenges in interference mitigation, power consumption, and real-time configuration. A novel network architecture utilizing multiple access points (APs), STAR-RISs, and NOMA is proposed for indoor communication.
  arXiv  Detail & Related papers  (2024-06-19T07:17:04Z)
• Open RAN LSTM Traffic Prediction and Slice Management using Deep Reinforcement Learning [7.473473066047965]
  This paper introduces a novel approach to ORAN slicing using distributed deep reinforcement learning (DDRL) Simulation results demonstrate significant improvements in network performance, particularly in reducing violations. This emphasizes the importance of using the prediction rApp and distributed actors' information jointly as part of a dynamic xApp.
  arXiv  Detail & Related papers  (2024-01-12T22:43:07Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Evolutionary Deep Reinforcement Learning for Dynamic Slice Management in O-RAN [11.464582983164991]
  New open radio access network (O-RAN) with distinguishing features such as flexible design, disaggregated virtual and programmable components, and intelligent closed-loop control was developed. O-RAN slicing is being investigated as a critical strategy for ensuring network quality of service (QoS) in the face of changing circumstances. This paper introduces a novel framework able to manage the network slices through provisioned resources intelligently.
  arXiv  Detail & Related papers  (2022-08-30T17:00:53Z)
• Multi-Objective Provisioning of Network Slices using Deep Reinforcement Learning [5.074839768784803]
  A real-time Network Slice Provisioning (NSP) problem is modeled as an online Multi-Objective Programming Optimization (MOIPO) problem. We approximate the solution of the MOIPO problem by applying the Proximal Policy Optimization (PPO) method to the traffic demand prediction. Our simulation results show the effectiveness of the proposed method compared to the state-of-the-art MOIPO solvers with a lower SLA violation rate and network operation cost.
  arXiv  Detail & Related papers  (2022-07-27T23:04:22Z)
• State-Augmented Learnable Algorithms for Resource Management in Wireless Networks [124.89036526192268]
  We propose a state-augmented algorithm for solving resource management problems in wireless networks. We show that the proposed algorithm leads to feasible and near-optimal RRM decisions.
  arXiv  Detail & Related papers  (2022-07-05T18:02:54Z)
• Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks [82.02891936174221]
  Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach. In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
  arXiv  Detail & Related papers  (2021-11-23T18:24:47Z)
• CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing [19.990451009223573]
  Network slicing is proposed as a promising solution for resource utilization in 5G and future networks. We formulate the problem as a Constrained Markov Decision Process (CMDP) without knowing models and hidden structures. We propose to solve the problem using CLARA, a Constrained reinforcement LeArning based Resource Allocation algorithm.
  arXiv  Detail & Related papers  (2021-11-16T11:54:09Z)
• Offline Contextual Bandits for Wireless Network Optimization [107.24086150482843]
  In this paper, we investigate how to learn policies that can automatically adjust the configuration parameters of every cell in the network in response to the changes in the user demand. Our solution combines existent methods for offline learning and adapts them in a principled way to overcome crucial challenges arising in this context.
  arXiv  Detail & Related papers  (2021-11-11T11:31:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.