OnSlicing: Online End-to-End Network Slicing with Reinforcement Learning

• URL: http://arxiv.org/abs/2111.01616v1
• Date: Tue, 2 Nov 2021 14:25:58 GMT
• Title: OnSlicing: Online End-to-End Network Slicing with Reinforcement Learning
• Authors: Qiang Liu and Nakjung Choi and Tao Han
• Abstract summary: We propose OnSlicing, an online end-to-end network slicing system, to achieve minimal resource usage while satisfying slices' SLA. OnSlicing allows individualized learning for each slice and maintains its SLA by using a novel constraint-aware policy update method. We implement OnSlicing on an end-to-end slicing testbed designed based on OpenAirInterface with both 4G LTE and 5G NR, OpenDayLight SDN platform, and OpenAir-CN core network.
• Score: 11.420934703203878
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Network slicing allows mobile network operators to virtualize infrastructures and provide customized slices for supporting various use cases with heterogeneous requirements. Online deep reinforcement learning (DRL) has shown promising potential in solving network problems and eliminating the simulation-to-reality discrepancy. Optimizing cross-domain resources with online DRL is, however, challenging, as the random exploration of DRL violates the service level agreement (SLA) of slices and resource constraints of infrastructures. In this paper, we propose OnSlicing, an online end-to-end network slicing system, to achieve minimal resource usage while satisfying slices' SLA. OnSlicing allows individualized learning for each slice and maintains its SLA by using a novel constraint-aware policy update method and proactive baseline switching mechanism. OnSlicing complies with resource constraints of infrastructures by using a unique design of action modification in slices and parameter coordination in infrastructures. OnSlicing further mitigates the poor performance of online learning during the early learning stage by offline imitating a rule-based solution. Besides, we design four new domain managers to enable dynamic resource configuration in radio access, transport, core, and edge networks, respectively, at a timescale of subseconds. We implement OnSlicing on an end-to-end slicing testbed designed based on OpenAirInterface with both 4G LTE and 5G NR, OpenDayLight SDN platform, and OpenAir-CN core network. The experimental results show that OnSlicing achieves 61.3% usage reduction as compared to the rule-based solution and maintains nearly zero violation (0.06%) throughout the online learning phase. As online learning is converged, OnSlicing reduces 12.5% usage without any violations as compared to the state-of-the-art online DRL solution.

Related papers

• Resilient UAV Trajectory Planning via Few-Shot Meta-Offline Reinforcement Learning [5.771885923067511]
  This work proposes a novel, resilient, few-shot meta-offline RL algorithm combining offline RL and model-agnostic meta-learning. We show that the proposed few-shot meta-offline RL algorithm converges faster than baseline schemes. It is the only algorithm that can achieve optimal joint AoI and transmission power using an offline dataset.
  arXiv  Detail & Related papers  (2025-02-03T11:39:12Z)
• Joint Admission Control and Resource Allocation of Virtual Network Embedding via Hierarchical Deep Reinforcement Learning [69.00997996453842]
  We propose a deep Reinforcement Learning approach to learn a joint Admission Control and Resource Allocation policy for virtual network embedding. We show that HRL-ACRA outperforms state-of-the-art baselines in terms of both the acceptance ratio and long-term average revenue.
  arXiv  Detail & Related papers  (2024-06-25T07:42:30Z)
• Action-Quantized Offline Reinforcement Learning for Robotic Skill Learning [68.16998247593209]
  offline reinforcement learning (RL) paradigm provides recipe to convert static behavior datasets into policies that can perform better than the policy that collected the data. In this paper, we propose an adaptive scheme for action quantization. We show that several state-of-the-art offline RL methods such as IQL, CQL, and BRAC improve in performance on benchmarks when combined with our proposed discretization scheme.
  arXiv  Detail & Related papers  (2023-10-18T06:07:10Z)
• Federated Reinforcement Learning for Resource Allocation in V2X Networks [46.6256432514037]
  Resource allocation significantly impacts the performance of vehicle-to-everything (V2X) networks. Most existing algorithms for resource allocation are based on optimization or machine learning. In this paper, we explore resource allocation in a V2X network under the framework of federated reinforcement learning.
  arXiv  Detail & Related papers  (2023-10-15T15:26:54Z)
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
  Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
  arXiv  Detail & Related papers  (2023-03-16T21:06:13Z)
• Atlas: Automate Online Service Configuration in Network Slicing [11.968135071159304]
  We propose Atlas, an online network slicing system, which automates the service configuration of slices. First, we design a learning-based simulator to reduce the sim-to-real discrepancy. Second, we offline train the policy in the augmented simulator via a novel offline algorithm with a Bayesian neural network and parallel Thompson sampling. Third, we online learn the policy in real networks with a novel online algorithm with safe exploration and Gaussian process regression.
  arXiv  Detail & Related papers  (2022-10-30T17:51:00Z)
• Boosting Offline Reinforcement Learning via Data Rebalancing [104.3767045977716]
  offline reinforcement learning (RL) is challenged by the distributional shift between learning policies and datasets. We propose a simple yet effective method to boost offline RL algorithms based on the observation that resampling a dataset keeps the distribution support unchanged. We dub our method ReD (Return-based Data Rebalance), which can be implemented with less than 10 lines of code change and adds negligible running time.
  arXiv  Detail & Related papers  (2022-10-17T16:34:01Z)
• Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient [62.660451283548724]
  ResiNet is a reinforcement learning framework to discover resilient network topologies against various disasters and attacks. We show that ResiNet achieves a near-optimal resilience gain on multiple graphs while balancing the utility, with a large margin compared to existing approaches.
  arXiv  Detail & Related papers  (2021-10-18T06:14:28Z)
• A Heuristically Assisted Deep Reinforcement Learning Approach for Network Slice Placement [0.7885276250519428]
  We introduce a hybrid placement solution based on Deep Reinforcement Learning (DRL) and a dedicated optimization based on the Power of Two Choices principle. The proposed Heuristically-Assisted DRL (HA-DRL) allows to accelerate the learning process and gain in resource usage when compared against other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-05-14T10:04:17Z)
• Dynamic RAN Slicing for Service-Oriented Vehicular Networks via Constrained Learning [40.5603189901241]
  We investigate a radio access network (RAN) slicing problem for Internet of vehicles (IoV) services with different quality of service (QoS) requirements. A dynamic RAN slicing framework is presented to dynamically allocate radio spectrum and computing resource. We show that the RAWS effectively reduces the system cost while satisfying requirements with a high probability, as compared with benchmarks.
  arXiv  Detail & Related papers  (2020-12-03T15:08:38Z)
• DeepSlicing: Deep Reinforcement Learning Assisted Resource Allocation for Network Slicing [20.723527476555574]
  Network slicing enables multiple virtual networks run on the same physical infrastructure to support various use cases in 5G and beyond. These use cases have very diverse network resource demands, e.g., communication and computation, and various performance metrics such as latency and throughput. We propose DeepSlicing that integrates the alternating direction method of multipliers (ADMM) and deep reinforcement learning (DRL)
  arXiv  Detail & Related papers  (2020-08-17T20:52:19Z)
• Deep Adaptive Inference Networks for Single Image Super-Resolution [72.7304455761067]
  Single image super-resolution (SISR) has witnessed tremendous progress in recent years owing to the deployment of deep convolutional neural networks (CNNs) In this paper, we take a step forward to address this issue by leveraging the adaptive inference networks for deep SISR (AdaDSR) Our AdaDSR involves an SISR model as backbone and a lightweight adapter module which takes image features and resource constraint as input and predicts a map of local network depth.
  arXiv  Detail & Related papers  (2020-04-08T10:08:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.