The Greatest Teacher, Failure is: Using Reinforcement Learning for SFC Placement Based on Availability and Energy Consumption

• URL: http://arxiv.org/abs/2010.05711v2
• Date: Wed, 18 Nov 2020 22:40:46 GMT
• Title: The Greatest Teacher, Failure is: Using Reinforcement Learning for SFC Placement Based on Availability and Energy Consumption
• Authors: Guto Leoni Santos, Theo Lynn, Judith Kelner, Patricia Takako Endo
• Abstract summary: Telecommunication operators are deploying increasingly complex service function chains (SFCs) This paper proposes an availability- and energy-aware solution based on reinforcement learning (RL) Two policy-aware RL algorithms, Advantage Actor-Critic (A2C) and Proximal Policy optimisation (PPO2), are compared using simulations of a ground truth network topology based on the Rede Nacional de Ensino e Pesquisa (RNP) Network, Brazil's National Teaching and Research Network backbone.
• Score: 0.3441021278275805
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Software defined networking (SDN) and network functions virtualisation (NFV) are making networks programmable and consequently much more flexible and agile. To meet service level agreements, achieve greater utilisation of legacy networks, faster service deployment, and reduce expenditure, telecommunications operators are deploying increasingly complex service function chains (SFCs). Notwithstanding the benefits of SFCs, increasing heterogeneity and dynamism from the cloud to the edge introduces significant SFC placement challenges, not least adding or removing network functions while maintaining availability, quality of service, and minimising cost. In this paper, an availability- and energy-aware solution based on reinforcement learning (RL) is proposed for dynamic SFC placement. Two policy-aware RL algorithms, Advantage Actor-Critic (A2C) and Proximal Policy Optimisation (PPO2), are compared using simulations of a ground truth network topology based on the Rede Nacional de Ensino e Pesquisa (RNP) Network, Brazil's National Teaching and Research Network backbone. The simulation results showed that PPO2 generally outperformed A2C and a greedy approach both in terms of acceptance rate and energy consumption. A2C outperformed PPO2 only in the scenario where network servers had a greater number of computing resources.

Related papers

• Federated Reinforcement Learning for Resource Allocation in V2X Networks [46.6256432514037]
  Resource allocation significantly impacts the performance of vehicle-to-everything (V2X) networks. Most existing algorithms for resource allocation are based on optimization or machine learning. In this paper, we explore resource allocation in a V2X network under the framework of federated reinforcement learning.
  arXiv  Detail & Related papers  (2023-10-15T15:26:54Z)
• Inter-Cell Network Slicing With Transfer Learning Empowered Multi-Agent Deep Reinforcement Learning [6.523367518762879]
  Network slicing enables operators to efficiently support diverse applications on a common physical infrastructure. The ever-increasing densification of network deployment leads to complex and non-trivial inter-cell interference. We develop a DIRP algorithm with multiple deep reinforcement learning (DRL) agents to cooperatively optimize resource partition in individual cells.
  arXiv  Detail & Related papers  (2023-06-20T14:14:59Z)
• Multi-Agent Reinforcement Learning for Network Routing in Integrated Access Backhaul Networks [0.0]
  We aim to maximize packet arrival ratio while minimizing their latency in IAB networks. To solve this problem, we develop a multi-agent partially observed Markov decision process (POMD) We show that A2C outperforms other reinforcement learning algorithms, leading to increased network efficiency and reduced selfish agent behavior.
  arXiv  Detail & Related papers  (2023-05-12T13:03:26Z)
• Multi-Objective Provisioning of Network Slices using Deep Reinforcement Learning [5.074839768784803]
  A real-time Network Slice Provisioning (NSP) problem is modeled as an online Multi-Objective Programming Optimization (MOIPO) problem. We approximate the solution of the MOIPO problem by applying the Proximal Policy Optimization (PPO) method to the traffic demand prediction. Our simulation results show the effectiveness of the proposed method compared to the state-of-the-art MOIPO solvers with a lower SLA violation rate and network operation cost.
  arXiv  Detail & Related papers  (2022-07-27T23:04:22Z)
• On Jointly Optimizing Partial Offloading and SFC Mapping: A Cooperative Dual-agent Deep Reinforcement Learning Approach [8.168647937560504]
  This paper studies the partial offloading and SFC mapping joint optimization (POSMJO) problem in an computation-enabled MEC system. The objective is to minimize the average cost in the long term which is a combination of execution delay, MD's energy consumption, and usage charge for edge computing. We propose a cooperative dual-agent deep reinforcement learning (CDADRL) algorithm, where we design a framework enabling interaction between two agents.
  arXiv  Detail & Related papers  (2022-05-20T02:00:53Z)
• Federated Learning over Wireless IoT Networks with Optimized Communication and Resources [98.18365881575805]
  Federated learning (FL) as a paradigm of collaborative learning techniques has obtained increasing research attention. It is of interest to investigate fast responding and accurate FL schemes over wireless systems. We show that the proposed communication-efficient federated learning framework converges at a strong linear rate.
  arXiv  Detail & Related papers  (2021-10-22T13:25:57Z)
• A Generic Visualization Approach for Convolutional Neural Networks [48.30883603606862]
  We formulate attention visualization as a constrained optimization problem. We leverage the unit L2-Norm constraint as an attention filter (L2-CAF) to localize attention in both classification and retrieval networks.
  arXiv  Detail & Related papers  (2020-07-19T18:46:56Z)
• Cognitive Radio Network Throughput Maximization with Deep Reinforcement Learning [58.44609538048923]
  Radio Frequency powered Cognitive Radio Networks (RF-CRN) are likely to be the eyes and ears of upcoming modern networks such as Internet of Things (IoT) To be considered autonomous, the RF-powered network entities need to make decisions locally to maximize the network throughput under the uncertainty of any network environment. In this paper, deep reinforcement learning is proposed to overcome the shortcomings and allow a wireless gateway to derive an optimal policy to maximize network throughput.
  arXiv  Detail & Related papers  (2020-07-07T01:49:07Z)
• Using Reinforcement Learning to Allocate and Manage Service Function Chains in Cellular Networks [0.456877715768796]
  We propose the use of reinforcement learning to deploy a service function chain (SFC) of cellular network service and manage the network virtual functions (VNFs) The main purpose is to reduce the number of lost packets taking into account the energy consumption of the servers. Preliminary results show that the agent is able to allocate the SFC and manage the VNFs, reducing the number of lost packets.
  arXiv  Detail & Related papers  (2020-06-12T17:38:23Z)
• Deep Learning for Radio Resource Allocation with Diverse Quality-of-Service Requirements in 5G [53.23237216769839]
  We develop a deep learning framework to approximate the optimal resource allocation policy for base stations. We find that a fully-connected neural network (NN) cannot fully guarantee the requirements due to the approximation errors and quantization errors of the numbers of subcarriers. Considering that the distribution of wireless channels and the types of services in the wireless networks are non-stationary, we apply deep transfer learning to update NNs in non-stationary wireless networks.
  arXiv  Detail & Related papers  (2020-03-29T04:48:22Z)
• ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions [76.05981545084738]
  We propose several ideas for enhancing a binary network to close its accuracy gap from real-valued networks without incurring any additional computational cost. We first construct a baseline network by modifying and binarizing a compact real-valued network with parameter-free shortcuts. We show that the proposed ReActNet outperforms all the state-of-the-arts by a large margin.
  arXiv  Detail & Related papers  (2020-03-07T02:12:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.