Accelerating Deep Reinforcement Learning for Digital Twin Network Optimization with Evolutionary Strategies

• URL: http://arxiv.org/abs/2202.00360v1
• Date: Tue, 1 Feb 2022 11:56:55 GMT
• Title: Accelerating Deep Reinforcement Learning for Digital Twin Network Optimization with Evolutionary Strategies
• Authors: Carlos G\"uemes-Palau (1), Paul Almasan (1), Shihan Xiao (2), Xiangle Cheng (2), Xiang Shi (2), Pere Barlet-Ros (1), Albert Cabellos-Aparicio (1) ((1) Barcelona Neural Networking Center, Universitat Polit\`ecnica de Catalunya, Spain (2) Network Technology Lab., Huawei Technologies Co., Ltd.)
• Abstract summary: The community proposed the Digital Twin Networks (DTN) as a key enabler of efficient network management. Deep Reinforcement Learning (DRL) showed a high performance when applied to solve network optimization problems. In this paper, we explore the use of Evolutionary Strategies (ES) to train DRL agents for solving a routing optimization problem.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The recent growth of emergent network applications (e.g., satellite networks, vehicular networks) is increasing the complexity of managing modern communication networks. As a result, the community proposed the Digital Twin Networks (DTN) as a key enabler of efficient network management. Network operators can leverage the DTN to perform different optimization tasks (e.g., Traffic Engineering, Network Planning). Deep Reinforcement Learning (DRL) showed a high performance when applied to solve network optimization problems. In the context of DTN, DRL can be leveraged to solve optimization problems without directly impacting the real-world network behavior. However, DRL scales poorly with the problem size and complexity. In this paper, we explore the use of Evolutionary Strategies (ES) to train DRL agents for solving a routing optimization problem. The experimental results show that ES achieved a training time speed-up of 128 and 6 for the NSFNET and GEANT2 topologies respectively.

Related papers

• DRL Optimization Trajectory Generation via Wireless Network Intent-Guided Diffusion Models for Optimizing Resource Allocation [58.62766376631344]
  We propose a customized wireless network intent (WNI-G) model to address different state variations of wireless communication networks. Extensive simulation achieves greater stability in spectral efficiency and variations of traditional DRL models in dynamic communication systems.
  arXiv  Detail & Related papers  (2024-10-18T14:04:38Z)
• Intelligent Routing Algorithm over SDN: Reusable Reinforcement Learning Approach [1.799933345199395]
  We develop a reusable RL-aware, reusable routing algorithm, RLSR-Routing over SDN. Our algorithm shows better performance in terms of load balancing than the traditional approaches. It also has faster convergence than the non-reusable RL approach when finding paths for multiple traffic demands.
  arXiv  Detail & Related papers  (2024-09-23T17:15:24Z)
• Multiobjective Vehicle Routing Optimization with Time Windows: A Hybrid Approach Using Deep Reinforcement Learning and NSGA-II [52.083337333478674]
  This paper proposes a weight-aware deep reinforcement learning (WADRL) approach designed to address the multiobjective vehicle routing problem with time windows (MOVRPTW) The Non-dominated sorting genetic algorithm-II (NSGA-II) method is then employed to optimize the outcomes produced by the WADRL.
  arXiv  Detail & Related papers  (2024-07-18T02:46:06Z)
• A Deep Reinforcement Learning Approach for Adaptive Traffic Routing in Next-gen Networks [1.1586742546971471]
  Next-gen networks require automation and adaptively adjust network configuration based on traffic dynamics. Traditional techniques that decide traffic policies are usually based on hand-crafted programming optimization and algorithms. We develop a deep reinforcement learning (DRL) approach for adaptive traffic routing.
  arXiv  Detail & Related papers  (2024-02-07T01:48:29Z)
• Solving Large-scale Spatial Problems with Convolutional Neural Networks [88.31876586547848]
  We employ transfer learning to improve training efficiency for large-scale spatial problems. We propose that a convolutional neural network (CNN) can be trained on small windows of signals, but evaluated on arbitrarily large signals with little to no performance degradation.
  arXiv  Detail & Related papers  (2023-06-14T01:24:42Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Multi-agent Reinforcement Learning with Graph Q-Networks for Antenna Tuning [60.94661435297309]
  The scale of mobile networks makes it challenging to optimize antenna parameters using manual intervention or hand-engineered strategies. We propose a new multi-agent reinforcement learning algorithm to optimize mobile network configurations globally. We empirically demonstrate the performance of the algorithm on an antenna tilt tuning problem and a joint tilt and power control problem in a simulated environment.
  arXiv  Detail & Related papers  (2023-01-20T17:06:34Z)
• A Reinforcement Learning Approach to Optimize Available Network Bandwidth Utilization [3.254879465902239]
  We present a novel approach to finding the optimum number of parallel TCP streams using deep reinforcement learning (RL) Our RL-based algorithm can find near-optimal solutions 40% faster while achieving up to 15% higher throughput.
  arXiv  Detail & Related papers  (2022-11-22T02:00:05Z)
• RDRN: Recursively Defined Residual Network for Image Super-Resolution [58.64907136562178]
  Deep convolutional neural networks (CNNs) have obtained remarkable performance in single image super-resolution. We propose a novel network architecture which utilizes attention blocks efficiently.
  arXiv  Detail & Related papers  (2022-11-17T11:06:29Z)
• Learning an Adaptive Forwarding Strategy for Mobile Wireless Networks: Resource Usage vs. Latency [2.608874253011]
  We use deep reinforcement learning to learn a scalable and generalizable single-copy routing strategy for mobile networks. Our results show our learned single-copy routing strategy outperforms all other strategies in terms of delay except for the optimal strategy.
  arXiv  Detail & Related papers  (2022-07-23T01:17:23Z)
• ENERO: Efficient Real-Time Routing Optimization [2.830334160074889]
  Traffic Engineering (TE) solutions must be able to achieve high performance real-time network operation. Current TE technologies rely on hand-crafteds or computationally expensive solvers. We propose Enero, an efficient real-time TE engine.
  arXiv  Detail & Related papers  (2021-09-22T17:53:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.