Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey

• URL: http://arxiv.org/abs/2110.13484v1
• Date: Tue, 26 Oct 2021 08:26:55 GMT
• Title: Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey
• Authors: Tianxu Li, Kun Zhu, Nguyen Cong Luong, Dusit Niyato, Qihui Wu, Yang Zhang, Bing Chen
• Abstract summary: Multi-agent Reinforcement Learning (MARL) allows each network entity to learn its optimal policy by observing not only the environments, but also other entities' policies. MARL can significantly improve the learning efficiency of the network entities, and it has been recently used to solve various issues in the emerging networks.
• Score: 45.805062677919
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Future Internet involves several emerging technologies such as 5G and beyond 5G networks, vehicular networks, unmanned aerial vehicle (UAV) networks, and Internet of Things (IoTs). Moreover, future Internet becomes heterogeneous and decentralized with a large number of involved network entities. Each entity may need to make its local decision to improve the network performance under dynamic and uncertain network environments. Standard learning algorithms such as single-agent Reinforcement Learning (RL) or Deep Reinforcement Learning (DRL) have been recently used to enable each network entity as an agent to learn an optimal decision-making policy adaptively through interacting with the unknown environments. However, such an algorithm fails to model the cooperations or competitions among network entities, and simply treats other entities as a part of the environment that may result in the non-stationarity issue. Multi-agent Reinforcement Learning (MARL) allows each network entity to learn its optimal policy by observing not only the environments, but also other entities' policies. As a result, MARL can significantly improve the learning efficiency of the network entities, and it has been recently used to solve various issues in the emerging networks. In this paper, we thus review the applications of MARL in the emerging networks. In particular, we provide a tutorial of MARL and a comprehensive survey of applications of MARL in next generation Internet. In particular, we first introduce single-agent RL and MARL. Then, we review a number of applications of MARL to solve emerging issues in future Internet. The issues consist of network access, transmit power control, computation offloading, content caching, packet routing, trajectory design for UAV-aided networks, and network security issues.

Related papers

• Towards Intelligent Network Management: Leveraging AI for Network Service Detection [0.0]
  This study focuses on leveraging Machine Learning methodologies to create an advanced network traffic classification system. We introduce a novel data-driven approach that excels in identifying various network service types in real-time. Our system demonstrates a remarkable accuracy in distinguishing the network services.
  arXiv  Detail & Related papers  (2023-10-14T16:06:11Z)
• Emergent Communication in Multi-Agent Reinforcement Learning for Future Wireless Networks [30.678152524314225]
  Multi-agent reinforcement learning with emergent communication (EC-MARL) is a promising solution to address high dimensional continuous control problems. This paper articulates the importance of EC-MARL within the context of future 6G wireless networks, which imbues autonomous decision-making capabilities into network entities.
  arXiv  Detail & Related papers  (2023-09-12T07:40:53Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Distributed Learning in Wireless Networks: Recent Progress and Future Challenges [170.35951727508225]
  Next-generation wireless networks will enable many machine learning (ML) tools and applications to analyze various types of data collected by edge devices. Distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges. This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks.
  arXiv  Detail & Related papers  (2021-04-05T20:57:56Z)
• Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial [29.76086936463468]
  This tutorial focuses on the role of Deep Reinforcement Learning (DRL) with an emphasis on deep Multi-Agent Reinforcement Learning (MARL) for AI-enabled 6G networks. The first part of this paper will present a clear overview of the mathematical frameworks for single-agent RL and MARL. We provide a selective description of RL algorithms such as Model-Based RL (MBRL) and cooperative MARL.
  arXiv  Detail & Related papers  (2020-11-06T22:12:40Z)
• A Tutorial on Ultra-Reliable and Low-Latency Communications in 6G: Integrating Domain Knowledge into Deep Learning [115.75967665222635]
  Ultra-reliable and low-latency communications (URLLC) will be central for the development of various emerging mission-critical applications. Deep learning algorithms have been considered as promising ways of developing enabling technologies for URLLC in future 6G networks. This tutorial illustrates how domain knowledge can be integrated into different kinds of deep learning algorithms for URLLC.
  arXiv  Detail & Related papers  (2020-09-13T14:53:01Z)
• From Federated to Fog Learning: Distributed Machine Learning over Heterogeneous Wireless Networks [71.23327876898816]
  Federated learning has emerged as a technique for training ML models at the network edge by leveraging processing capabilities across the nodes that collect the data. We advocate a new learning paradigm called fog learning which will intelligently distribute ML model training across the continuum of nodes from edge devices to cloud servers.
  arXiv  Detail & Related papers  (2020-06-07T05:11:18Z)
• NetML: A Challenge for Network Traffic Analytics [16.8001000840057]
  We release three open datasets containing almost 1.3M labeled flows in total. We focus on broad aspects in network traffic analysis, including both malware detection and application classification. As we continue to grow NetML, we expect the datasets to serve as a common platform for AI driven, reproducible research on network flow analytics.
  arXiv  Detail & Related papers  (2020-04-25T01:12:17Z)
• Distributed Learning in Ad-Hoc Networks: A Multi-player Multi-armed Bandit Framework [0.0]
  Next-generation networks are expected to be ultra-dense with a very high peak rate but relatively lower expected traffic per user. To overcome this problem, cognitive ad-hoc networks (CAHN) that share spectrum with other networks are being envisioned. We discuss state-of-the-art multi-armed multi-player bandit based distributed learning algorithms that allow users to adapt to the environment.
  arXiv  Detail & Related papers  (2020-03-06T18:11:47Z)
• Deep Learning for Ultra-Reliable and Low-Latency Communications in 6G Networks [84.2155885234293]
  We first summarize how to apply data-driven supervised deep learning and deep reinforcement learning in URLLC. To address these open problems, we develop a multi-level architecture that enables device intelligence, edge intelligence, and cloud intelligence for URLLC.
  arXiv  Detail & Related papers  (2020-02-22T14:38:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.