A Q-Learning-based Approach for Distributed Beam Scheduling in mmWave Networks

• URL: http://arxiv.org/abs/2110.08704v1
• Date: Sun, 17 Oct 2021 02:58:13 GMT
• Title: A Q-Learning-based Approach for Distributed Beam Scheduling in mmWave Networks
• Authors: Xiang Zhang, Shamik Sarkar, Arupjyoti Bhuyan, Sneha Kumar Kasera, Mingyue Ji
• Abstract summary: We consider the problem of distributed downlink beam scheduling and power allocation for millimeter-Wave (mmWave) cellular networks. Multiple base stations belonging to different service operators share the same unlicensed spectrum with no central coordination or cooperation among them. We propose a distributed scheduling approach to power allocation and adaptation for efficient interference management over the shared spectrum by modeling each BS as an independent Q-learning agent.
• Score: 18.22250038264899
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of distributed downlink beam scheduling and power allocation for millimeter-Wave (mmWave) cellular networks where multiple base stations (BSs) belonging to different service operators share the same unlicensed spectrum with no central coordination or cooperation among them. Our goal is to design efficient distributed beam scheduling and power allocation algorithms such that the network-level payoff, defined as the weighted sum of the total throughput and a power penalization term, can be maximized. To this end, we propose a distributed scheduling approach to power allocation and adaptation for efficient interference management over the shared spectrum by modeling each BS as an independent Q-learning agent. As a baseline, we compare the proposed approach to the state-of-the-art non-cooperative game-based approach which was previously developed for the same problem. We conduct extensive experiments under various scenarios to verify the effect of multiple factors on the performance of both approaches. Experiment results show that the proposed approach adapts well to different interference situations by learning from experience and can achieve higher payoff than the game-based approach. The proposed approach can also be integrated into our previously developed Lyapunov stochastic optimization framework for the purpose of network utility maximization with optimality guarantee. As a result, the weights in the payoff function can be automatically and optimally determined by the virtual queue values from the sub-problems derived from the Lyapunov optimization framework.

Related papers

• Non-iterative Optimization of Trajectory and Radio Resource for Aerial Network [7.824710236769593]
  We address a joint trajectory planning, user association, resource allocation, and power control problem in the aerial IoT network. Our framework can incorporate various trajectory planning algorithms such as the genetic, tree search, and reinforcement learning.
  arXiv  Detail & Related papers  (2024-05-02T14:21:29Z)
• Federated Multi-Level Optimization over Decentralized Networks [55.776919718214224]
  We study the problem of distributed multi-level optimization over a network, where agents can only communicate with their immediate neighbors. We propose a novel gossip-based distributed multi-level optimization algorithm that enables networked agents to solve optimization problems at different levels in a single timescale. Our algorithm achieves optimal sample complexity, scaling linearly with the network size, and demonstrates state-of-the-art performance on various applications.
  arXiv  Detail & Related papers  (2023-10-10T00:21:10Z)
• Predictive GAN-powered Multi-Objective Optimization for Hybrid Federated Split Learning [56.125720497163684]
  We propose a hybrid federated split learning framework in wireless networks. We design a parallel computing scheme for model splitting without label sharing, and theoretically analyze the influence of the delayed gradient caused by the scheme on the convergence speed.
  arXiv  Detail & Related papers  (2022-09-02T10:29:56Z)
• Distributed Proximal Policy Optimization for Contention-Based Spectrum Access [40.99534735484468]
  We develop a novel distributed implementation of a policy gradient method known as Proximal Policy Optimization. In each time slot, a base station uses information from spectrum sensing and reception quality to autonomously decide whether or not to transmit on a given resource. We find the proportional fairness reward accumulated by the policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold.
  arXiv  Detail & Related papers  (2021-10-07T00:54:03Z)
• Learning based E2E Energy Efficient in Joint Radio and NFV Resource Allocation for 5G and Beyond Networks [21.60295771932728]
  We formulate an optimization problem in which power and spectrum resources are allocated in the radio part. In the core part, the chaining, placement, and scheduling of functions are performed to ensure the efficiency of all users. A soft actor-critic deep learning (SAC-DRL) algorithm based on the maximum entropy framework is subsequently utilized to solve the above MDP.
  arXiv  Detail & Related papers  (2021-07-13T11:19:48Z)
• Optimal Power Allocation for Rate Splitting Communications with Deep Reinforcement Learning [61.91604046990993]
  This letter introduces a novel framework to optimize the power allocation for users in a Rate Splitting Multiple Access network. In the network, messages intended for users are split into different parts that are a single common part and respective private parts.
  arXiv  Detail & Related papers  (2021-07-01T06:32:49Z)
• Low-Latency Federated Learning over Wireless Channels with Differential Privacy [142.5983499872664]
  In federated learning (FL), model training is distributed over clients and local models are aggregated by a central server. In this paper, we aim to minimize FL training delay over wireless channels, constrained by overall training performance as well as each client's differential privacy (DP) requirement.
  arXiv  Detail & Related papers  (2021-06-20T13:51:18Z)
• Data-Driven Random Access Optimization in Multi-Cell IoT Networks with NOMA [78.60275748518589]
  Non-orthogonal multiple access (NOMA) is a key technology to enable massive machine type communications (mMTC) in 5G networks and beyond. In this paper, NOMA is applied to improve the random access efficiency in high-density spatially-distributed multi-cell wireless IoT networks. A novel formulation of random channel access management is proposed, in which the transmission probability of each IoT device is tuned to maximize the geometric mean of users' expected capacity.
  arXiv  Detail & Related papers  (2021-01-02T15:21:08Z)
• Decentralized MCTS via Learned Teammate Models [89.24858306636816]
  We present a trainable online decentralized planning algorithm based on decentralized Monte Carlo Tree Search. We show that deep learning and convolutional neural networks can be employed to produce accurate policy approximators.
  arXiv  Detail & Related papers  (2020-03-19T13:10:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.