MACC: Cross-Layer Multi-Agent Congestion Control with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2206.01972v1
• Date: Sat, 4 Jun 2022 12:02:35 GMT
• Title: MACC: Cross-Layer Multi-Agent Congestion Control with Deep Reinforcement Learning
• Authors: Jianing Bai, Tianhao Zhang, Guangming Xie
• Abstract summary: Congestion Control (CC) is core networking task to efficiently utilize network capacity. In this paper, we explore the performance of multi-agent reinforcement learning-based cross-layer congestion control algorithms.
• Score: 14.29757990259669
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Congestion Control (CC), as the core networking task to efficiently utilize network capacity, received great attention and widely used in various Internet communication applications such as 5G, Internet-of-Things, UAN, and more. Various CC algorithms have been proposed both on network and transport layers such as Active Queue Management (AQM) algorithm and Transmission Control Protocol (TCP) congestion control mechanism. But it is hard to model dynamic AQM/TCP system and cooperate two algorithms to obtain excellent performance under different communication scenarios. In this paper, we explore the performance of multi-agent reinforcement learning-based cross-layer congestion control algorithms and present cooperation performance of two agents, known as MACC (Multi-agent Congestion Control). We implement MACC in NS3. The simulation results show that our scheme outperforms other congestion control combination in terms of throughput and delay, etc. Not only does it proves that networking protocols based on multi-agent deep reinforcement learning is efficient for communication managing, but also verifies that networking area can be used as new playground for machine learning algorithms.

Related papers

• A Survey on Congestion Control and Scheduling for Multipath TCP: Machine Learning vs Classical Approaches [11.940300796370703]
  This paper reviews techniques to solve problems based on two main approaches; non data-driven (classical) and data-driven (Machine Learning) approaches. This paper compares these two approaches and highlights their strengths and weaknesses with a view to motivating future researchers in this exciting area of machine learning for communications.
  arXiv  Detail & Related papers  (2023-09-17T20:33:06Z)
• Learning to Sail Dynamic Networks: The MARLIN Reinforcement Learning Framework for Congestion Control in Tactical Environments [53.08686495706487]
  This paper proposes an RL framework that leverages an accurate and parallelizable emulation environment to reenact the conditions of a tactical network. We evaluate our RL learning framework by training a MARLIN agent in conditions replicating a bottleneck link transition between a Satellite Communication (SATCOM) and an UHF Wide Band (UHF) radio link.
  arXiv  Detail & Related papers  (2023-06-27T16:15:15Z)
• Multi-Agent Reinforcement Learning for Network Routing in Integrated Access Backhaul Networks [0.0]
  We aim to maximize packet arrival ratio while minimizing their latency in IAB networks. To solve this problem, we develop a multi-agent partially observed Markov decision process (POMD) We show that A2C outperforms other reinforcement learning algorithms, leading to increased network efficiency and reduced selfish agent behavior.
  arXiv  Detail & Related papers  (2023-05-12T13:03:26Z)
• Quantum Multi-Agent Actor-Critic Networks for Cooperative Mobile Access in Multi-UAV Systems [12.850810725666465]
  This paper proposes a novel algorithm, named quantum multi-agent actor-critic networks (QMACN) for autonomously constructing a robust mobile access system. The principles of a quantum computing (QC) are employed in our study to enhance the training process and inference capabilities of the UAVs involved.
  arXiv  Detail & Related papers  (2023-02-09T05:31:57Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet [48.96004919910818]
  We propose a multi-agent deep reinforcement learning (MADRL) based power control scheme for the HetNet. To promote cooperation among agents, we develop a penalty-based Q learning (PQL) algorithm for MADRL systems. In this way, an agent's policy can be learned by other agents more easily, resulting in a more efficient collaboration process.
  arXiv  Detail & Related papers  (2022-12-15T17:01:56Z)
• Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approach [56.84948632954274]
  This paper investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network. We propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network.
  arXiv  Detail & Related papers  (2021-12-20T15:45:28Z)
• Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks [82.02891936174221]
  Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach. In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
  arXiv  Detail & Related papers  (2021-11-23T18:24:47Z)
• Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning [20.385286762476436]
  Inefficient traffic control may cause numerous problems such as traffic congestion and energy waste. This paper proposes a novel multi-agent reinforcement learning method, named KS-DDPG, to achieve optimal control by enhancing the cooperation between traffic signals.
  arXiv  Detail & Related papers  (2021-04-20T12:53:08Z)
• Multi-Agent Reinforcement Learning in NOMA-aided UAV Networks for Cellular Offloading [59.32570888309133]
  A novel framework is proposed for cellular offloading with the aid of multiple unmanned aerial vehicles (UAVs) Non-orthogonal multiple access (NOMA) technique is employed at each UAV to further improve the spectrum efficiency of the wireless network. A mutual deep Q-network (MDQN) algorithm is proposed to jointly determine the optimal 3D trajectory and power allocation of UAVs.
  arXiv  Detail & Related papers  (2020-10-18T20:22:05Z)
• Online Multi-agent Reinforcement Learning for Decentralized Inverter-based Volt-VAR Control [3.260913246106564]
  The distributed Volt/Var control (VVC) methods have been widely studied for active distribution networks(ADNs) We propose an online multi-agent reinforcement learning and decentralized control framework (OLDC) for VVC.
  arXiv  Detail & Related papers  (2020-06-23T09:03:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.