Structured Reinforcement Learning for Delay-Optimal Data Transmission in Dense mmWave Networks

• URL: http://arxiv.org/abs/2404.16920v1
• Date: Thu, 25 Apr 2024 17:26:41 GMT
• Title: Structured Reinforcement Learning for Delay-Optimal Data Transmission in Dense mmWave Networks
• Authors: Shufan Wang, Guojun Xiong, Shichen Zhang, Huacheng Zeng, Jian Li, Shivendra Panwar,
• Abstract summary: We study the data packet transmission problem (mmDPT) in dense cell-free millimeter wave (mmWave) networks. Our objective is to minimize the average delay in the system due to APs' limited service capacity and wireless channels between APs and users.
• Score: 16.503911829712717
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the data packet transmission problem (mmDPT) in dense cell-free millimeter wave (mmWave) networks, i.e., users sending data packet requests to access points (APs) via uplinks and APs transmitting requested data packets to users via downlinks. Our objective is to minimize the average delay in the system due to APs' limited service capacity and unreliable wireless channels between APs and users. This problem can be formulated as a restless multi-armed bandits problem with fairness constraint (RMAB-F). Since finding the optimal policy for RMAB-F is intractable, existing learning algorithms are computationally expensive and not suitable for practical dynamic dense mmWave networks. In this paper, we propose a structured reinforcement learning (RL) solution for mmDPT by exploiting the inherent structure encoded in RMAB-F. To achieve this, we first design a low-complexity and provably asymptotically optimal index policy for RMAB-F. Then, we leverage this structure information to develop a structured RL algorithm called mmDPT-TS, which provably achieves an \tilde{O}(\sqrt{T}) Bayesian regret. More importantly, mmDPT-TS is computation-efficient and thus amenable to practical implementation, as it fully exploits the structure of index policy for making decisions. Extensive emulation based on data collected in realistic mmWave networks demonstrate significant gains of mmDPT-TS over existing approaches.

Related papers

• Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding [57.71603937699949]
  We study optimization guarantees, i.e., achieving near-zero training loss with the increase in the number of learning epochs. We show that the threshold on the number of training samples increases with the increase in the network width.
  arXiv  Detail & Related papers  (2023-09-12T13:03:47Z)
• Beam Management in Ultra-dense mmWave Network via Federated Reinforcement Learning: An Intelligent and Secure Approach [19.01563068819449]
  Key challenge of ultra-dense mmWave network (UDmmWave) is beam management due to high propagation delay limited beam coverage. In this paper, a novel beam management scheme is presented which can theoretically protect user privacy while reducing handoff cost.
  arXiv  Detail & Related papers  (2022-10-04T01:47:33Z)
• Learning OFDM Waveforms with PAPR and ACLR Constraints [15.423422040627331]
  We propose a learning-based method to design OFDM-based waveforms that satisfy selected constraints while maximizing an achievable information rate. We show that the end-to-end system is able to satisfy target PAPR and ACLR constraints and allows significant throughput gains.
  arXiv  Detail & Related papers  (2021-10-21T08:58:59Z)
• Unit-Modulus Wireless Federated Learning Via Penalty Alternating Minimization [64.76619508293966]
  Wireless federated learning (FL) is an emerging machine learning paradigm that trains a global parametric model from distributed datasets via wireless communications. This paper proposes a wireless FL framework, which uploads local model parameters and computes global model parameters via wireless communications.
  arXiv  Detail & Related papers  (2021-08-31T08:19:54Z)
• Low-Latency Federated Learning over Wireless Channels with Differential Privacy [142.5983499872664]
  In federated learning (FL), model training is distributed over clients and local models are aggregated by a central server. In this paper, we aim to minimize FL training delay over wireless channels, constrained by overall training performance as well as each client's differential privacy (DP) requirement.
  arXiv  Detail & Related papers  (2021-06-20T13:51:18Z)
• Ultra-Reliable Indoor Millimeter Wave Communications using Multiple Artificial Intelligence-Powered Intelligent Surfaces [115.85072043481414]
  We propose a novel framework for guaranteeing ultra-reliable millimeter wave (mmW) communications using multiple artificial intelligence (AI)-enabled reconfigurable intelligent surfaces (RISs) The use of multiple AI-powered RISs allows changing the propagation direction of the signals transmitted from a mmW access point (AP) Two centralized and distributed controllers are proposed to control the policies of the mmW AP and RISs.
  arXiv  Detail & Related papers  (2021-03-31T19:15:49Z)
• Adaptive Subcarrier, Parameter, and Power Allocation for Partitioned Edge Learning Over Broadband Channels [69.18343801164741]
  partitioned edge learning (PARTEL) implements parameter-server training, a well known distributed learning method, in wireless network. We consider the case of deep neural network (DNN) models which can be trained using PARTEL by introducing some auxiliary variables.
  arXiv  Detail & Related papers  (2020-10-08T15:27:50Z)
• Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications [54.610318402371185]
  Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. We minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences.
  arXiv  Detail & Related papers  (2020-05-25T01:42:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.