Multi-agent Attention Actor-Critic Algorithm for Load Balancing in Cellular Networks

• URL: http://arxiv.org/abs/2303.08003v1
• Date: Tue, 14 Mar 2023 15:51:33 GMT
• Title: Multi-agent Attention Actor-Critic Algorithm for Load Balancing in Cellular Networks
• Authors: Jikun Kang, Di Wu, Ju Wang, Ekram Hossain, Xue Liu, Gregory Dudek
• Abstract summary: In cellular networks, User Equipment (UE) handoff from one Base Station to another, giving rise to the load balancing problem among the BSs. This paper formulates the load balancing problem as a Markov game and proposes a Robust Multi-agent Attention Actor-Critic (Robust-MA3C) algorithm.
• Score: 33.72503214603868
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In cellular networks, User Equipment (UE) handoff from one Base Station (BS) to another, giving rise to the load balancing problem among the BSs. To address this problem, BSs can work collaboratively to deliver a smooth migration (or handoff) and satisfy the UEs' service requirements. This paper formulates the load balancing problem as a Markov game and proposes a Robust Multi-agent Attention Actor-Critic (Robust-MA3C) algorithm that can facilitate collaboration among the BSs (i.e., agents). In particular, to solve the Markov game and find a Nash equilibrium policy, we embrace the idea of adopting a nature agent to model the system uncertainty. Moreover, we utilize the self-attention mechanism, which encourages high-performance BSs to assist low-performance BSs. In addition, we consider two types of schemes, which can facilitate load balancing for both active UEs and idle UEs. We carry out extensive evaluations by simulations, and simulation results illustrate that, compared to the state-of-the-art MARL methods, Robust-\ours~scheme can improve the overall performance by up to 45%.

Related papers

• Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning [48.667697255912614]
  Mean-field reinforcement learning addresses the policy of a representative agent interacting with the infinite population of identical agents. We propose Safe-M$3$-UCRL, the first model-based mean-field reinforcement learning algorithm that attains safe policies even in the case of unknown transitions. Our algorithm effectively meets the demand in critical areas while ensuring service accessibility in regions with low demand.
  arXiv  Detail & Related papers  (2023-06-29T15:57:07Z)
• Learning Distributed and Fair Policies for Network Load Balancing as Markov Potentia Game [4.892398873024191]
  This paper investigates the network load balancing problem in data centers (DCs) where multiple load balancers (LBs) are deployed. The challenges of this problem consist of the heterogeneous processing architecture and dynamic environments. We formulate the multi-agent load balancing problem as a Markov potential game, with a carefully and properly designed workload distribution fairness as the potential function. A fully distributed MARL algorithm is proposed to approximate the Nash equilibrium of the game.
  arXiv  Detail & Related papers  (2022-06-03T08:29:02Z)
• Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning [63.83425382922157]
  Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
  arXiv  Detail & Related papers  (2022-03-26T20:37:14Z)
• Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation [93.52573037053449]
  H-MARL (Hallucinated Multi-Agent Reinforcement Learning) learns successful equilibrium policies after a few interactions with the environment. We demonstrate our approach experimentally on an autonomous driving simulation benchmark.
  arXiv  Detail & Related papers  (2022-03-14T17:24:03Z)
• Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center [4.141301293112916]
  This paper presents the network load balancing problem, a challenging real-world task for reinforcement learning methods. The cooperative network load balancing task is formulated as a Dec-POMDP problem, which naturally induces the MARL methods. To bridge the reality gap for applying learning-based methods, all methods are directly trained and evaluated on an emulation system.
  arXiv  Detail & Related papers  (2022-01-27T18:47:59Z)
• Towards Intelligent Load Balancing in Data Centers [0.5505634045241288]
  This paper proposes Aquarius to bridge the gap between machine learning and networking systems. It demonstrates its ability of conducting both offline data analysis and online model deployment in realistic systems.
  arXiv  Detail & Related papers  (2021-10-27T12:47:30Z)
• Softmax with Regularization: Better Value Estimation in Multi-Agent Reinforcement Learning [72.28520951105207]
  Overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning. We propose a novel regularization-based update scheme that penalizes large joint action-values deviating from a baseline. We show that our method provides a consistent performance improvement on a set of challenging StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2021-03-22T14:18:39Z)
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
  Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications. We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting. Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
  arXiv  Detail & Related papers  (2020-04-17T14:56:29Z)
• Value Variance Minimization for Learning Approximate Equilibrium in Aggregation Systems [8.140037969280716]
  We consider the problem of learning approximate equilibrium solutions (win-win) in aggregation systems. In this paper, we consider the problem of learning approximate equilibrium solutions (win-win) in aggregation systems so that individuals have an incentive to remain in the aggregation system.
  arXiv  Detail & Related papers  (2020-03-16T10:02:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.