Federated Neuroevolution O-RAN: Enhancing the Robustness of Deep Reinforcement Learning xApps

• URL: http://arxiv.org/abs/2506.12812v1
• Date: Sun, 15 Jun 2025 10:58:10 GMT
• Title: Federated Neuroevolution O-RAN: Enhancing the Robustness of Deep Reinforcement Learning xApps
• Authors: Mohammadreza Kouchaki, Aly Sabri Abdalla, Vuk Marojevic,
• Abstract summary: Reinforcement learning (RL) and its advanced form, deep RL (DRL) are increasingly employed for designing intelligent controllers, or xApps, to be deployed in the near-real time (near-RT) RIC.<n>These models often encounter local optima, which raise concerns about their reliability for RAN intelligent control.<n>We introduce Federated O- enabled Neuroevolution (NE-DRL) that deploys an NE-based xApp in parallel to the RAN controller xApps.<n>This NE-DRL xApp framework enables effective exploration and exploitation in the near-RT RIC without disrupting
• Score: 4.035007094168652
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The open radio access network (O-RAN) architecture introduces RAN intelligent controllers (RICs) to facilitate the management and optimization of the disaggregated RAN. Reinforcement learning (RL) and its advanced form, deep RL (DRL), are increasingly employed for designing intelligent controllers, or xApps, to be deployed in the near-real time (near-RT) RIC. These models often encounter local optima, which raise concerns about their reliability for RAN intelligent control. We therefore introduce Federated O-RAN enabled Neuroevolution (NE)-enhanced DRL (F-ONRL) that deploys an NE-based optimizer xApp in parallel to the RAN controller xApps. This NE-DRL xApp framework enables effective exploration and exploitation in the near-RT RIC without disrupting RAN operations. We implement the NE xApp along with a DRL xApp and deploy them on Open AI Cellular (OAIC) platform and present numerical results that demonstrate the improved robustness of xApps while effectively balancing the additional computational load.

Related papers

• Learning to Reason without External Rewards [100.27210579418562]
  Training large language models (LLMs) for complex reasoning via Reinforcement Learning with Verifiable Rewards (RLVR) is effective but limited by reliance on costly, domain-specific supervision.<n>We explore Reinforcement Learning from Internal Feedback (RLIF), a framework that enables LLMs to learn from intrinsic signals without external rewards or labeled data.<n>We propose Intuitor, an RLIF method that uses a model's own confidence, termed self-certainty, as its sole reward signal.
  arXiv  Detail & Related papers  (2025-05-26T07:01:06Z)
• LAMeTA: Intent-Aware Agentic Network Optimization via a Large AI Model-Empowered Two-Stage Approach [68.198383438396]
  We present LAMeTA, a Large AI Model (LAM)-empowered Two-stage Approach for intent-aware agentic network optimization.<n>First, we propose Intent-oriented Knowledge Distillation (IoKD), which efficiently distills intent-understanding capabilities.<n>Second, we develop Symbiotic Reinforcement Learning (SRL), integrating E-LAMs with a policy-based DRL framework.
  arXiv  Detail & Related papers  (2025-05-18T05:59:16Z)
• LLM-hRIC: LLM-empowered Hierarchical RAN Intelligent Control for O-RAN [56.94324843095396]
  This article introduces the LLM-empowered hierarchical RIC (LLM-hRIC) framework to improve the collaboration between RICs in radio access network (O-RAN)<n>The framework offers a strategic guidance to the near-real-time RIC (non-RT RIC) using global network information.<n>The RL-empowered near-RT RIC acts as an implementer, combining this guidance with local real-time data to make near-RT decisions.
  arXiv  Detail & Related papers  (2025-04-25T04:18:23Z)
• Enhancing Spectrum Efficiency in 6G Satellite Networks: A GAIL-Powered Policy Learning via Asynchronous Federated Inverse Reinforcement Learning [67.95280175998792]
  A novel adversarial imitation learning (GAIL)-powered policy learning approach is proposed for optimizing beamforming, spectrum allocation, and remote user equipment (RUE) association ins. We employ inverse RL (IRL) to automatically learn reward functions without manual tuning. We show that the proposed MA-AL method outperforms traditional RL approaches, achieving a $14.6%$ improvement in convergence and reward value.
  arXiv  Detail & Related papers  (2024-09-27T13:05:02Z)
• Generative AI for Deep Reinforcement Learning: Framework, Analysis, and Use Cases [60.30995339585003]
  Deep reinforcement learning (DRL) has been widely applied across various fields and has achieved remarkable accomplishments.<n>DRL faces certain limitations, including low sample efficiency and poor generalization.<n>We present how to leverage generative AI (GAI) to address these issues and enhance the performance of DRL algorithms.
  arXiv  Detail & Related papers  (2024-05-31T01:25:40Z)
• Safe and Accelerated Deep Reinforcement Learning-based O-RAN Slicing: A Hybrid Transfer Learning Approach [20.344810727033327]
  We propose and design a hybrid TL-aided approach to provide safe and accelerated convergence in DRL-based O-RAN slicing. The proposed hybrid approach shows at least: 7.7% and 20.7% improvements in the average initial reward value and the percentage of converged scenarios.
  arXiv  Detail & Related papers  (2023-09-13T18:58:34Z)
• Sparsity-Aware Intelligent Massive Random Access Control in Open RAN: A Reinforcement Learning Based Approach [61.74489383629319]
  Massive random access of devices in the emerging Open Radio Access Network (O-RAN) brings great challenge to the access control and management. reinforcement-learning (RL)-assisted scheme of closed-loop access control is proposed to preserve sparsity of access requests. Deep-RL-assisted SAUD is proposed to resolve highly complex environments with continuous and high-dimensional state and action spaces.
  arXiv  Detail & Related papers  (2023-03-05T12:25:49Z)
• Programmable and Customized Intelligence for Traffic Steering in 5G Networks Using Open RAN Architectures [16.48682480842328]
  5G and beyond mobile networks will support heterogeneous use cases at an unprecedented scale. Such fine-grained control of the Radio Access Network (RAN) is not possible with the current cellular architecture. We propose an open architecture with abstractions that enable closed-loop control and provide data-driven, and intelligent optimization of the RAN at the user level.
  arXiv  Detail & Related papers  (2022-09-28T15:31:06Z)
• Actor-Critic Network for O-RAN Resource Allocation: xApp Design, Deployment, and Analysis [3.8073142980733]
  Open Radio Access Network (O-RAN) has introduced an emerging RAN architecture that enables openness, intelligence, and automated control. The RAN Intelligent Controller (RIC) provides the platform to design and deploy RAN controllers. xApps are the applications which will take this responsibility by leveraging machine learning (ML) algorithms and acting in near-real time.
  arXiv  Detail & Related papers  (2022-09-26T19:12:18Z)
• FORLORN: A Framework for Comparing Offline Methods and Reinforcement Learning for Optimization of RAN Parameters [0.0]
  This paper introduces a new framework for benchmarking the performance of an RL agent in network environments simulated with ns-3. Within this framework, we demonstrate that an RL agent without domain-specific knowledge can learn how to efficiently adjust Radio Access Network (RAN) parameters to match offline optimization in static scenarios.
  arXiv  Detail & Related papers  (2022-09-08T12:58:09Z)
• ColO-RAN: Developing Machine Learning-based xApps for Open RAN Closed-loop Control on Programmable Experimental Platforms [22.260874168813647]
  ColO-RAN is the first publicly-available large-scale O-RAN testing framework with software-defined radios-in-the-loop. ColO-RAN enables ML research at scale using O-RAN components, programmable base stations, and a " wireless data factory" Extensive results from our first-of-its-kind large-scale evaluation highlight the benefits and challenges of DRL-based adaptive control.
  arXiv  Detail & Related papers  (2021-12-17T15:14:22Z)
• DriverGym: Democratising Reinforcement Learning for Autonomous Driving [75.91049219123899]
  We propose DriverGym, an open-source environment for developing reinforcement learning algorithms for autonomous driving. DriverGym provides access to more than 1000 hours of expert logged data and also supports reactive and data-driven agent behavior. The performance of an RL policy can be easily validated on real-world data using our extensive and flexible closed-loop evaluation protocol.
  arXiv  Detail & Related papers  (2021-11-12T11:47:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.