Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications

• URL: http://arxiv.org/abs/2005.11885v1
• Date: Mon, 25 May 2020 01:42:55 GMT
• Title: Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications
• Authors: Jiaye Lin, Yuze Zou, Xiaoru Dong, Shimin Gong, Dinh Thai Hoang, Dusit Niyato
• Abstract summary: Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. We minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences.
• Score: 54.610318402371185
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. In this paper, we minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. Due to uncertain channel conditions, we formulate a robust power minimization problem subject to the receiver's signal-to-noise ratio (SNR) requirement and the IRS's power budget constraint. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences. To improve the learning performance, we derive a convex approximation as a lower bound on the robust problem, which is integrated into the DRL framework and thus promoting a novel optimization-driven deep deterministic policy gradient (DDPG) approach. In particular, when the DDPG algorithm generates a part of the action (e.g., passive beamforming), we can use the model-based convex approximation to optimize the other part (e.g., active beamforming) of the action more efficiently. Our simulation results demonstrate that the optimization-driven DDPG algorithm can improve both the learning rate and reward performance significantly compared to the conventional model-free DDPG algorithm.

Related papers

• Enhancing Spectrum Efficiency in 6G Satellite Networks: A GAIL-Powered Policy Learning via Asynchronous Federated Inverse Reinforcement Learning [67.95280175998792]
  A novel adversarial imitation learning (GAIL)-powered policy learning approach is proposed for optimizing beamforming, spectrum allocation, and remote user equipment (RUE) association ins. We employ inverse RL (IRL) to automatically learn reward functions without manual tuning. We show that the proposed MA-AL method outperforms traditional RL approaches, achieving a $14.6%$ improvement in convergence and reward value.
  arXiv  Detail & Related papers  (2024-09-27T13:05:02Z)
• Reconfigurable Intelligent Surface Aided Vehicular Edge Computing: Joint Phase-shift Optimization and Multi-User Power Allocation [28.47670676456068]
  We introduce the use of Reconfigurable Intelligent Surfaces (RIS), which provide alternative communication pathways to assist vehicular communication. We propose an innovative deep reinforcement learning (DRL) framework that combines the Deep Deterministic Policy Gradient (DDPG) algorithm for optimizing RIS phase-shift coefficients and the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm for optimizing the power allocation of vehicle user (VU) Simulation results show that our proposed scheme outperforms the traditional centralized DDPG, Twin Delayed Deep Deterministic Policy Gradient (TD3) and some typical schemes.
  arXiv  Detail & Related papers  (2024-07-18T03:18:59Z)
• Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach [51.63921041249406]
  Non-orthogonal multiple access (NOMA) enables multiple users to share the same frequency band, and simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) deploying STAR-RIS indoors presents challenges in interference mitigation, power consumption, and real-time configuration. A novel network architecture utilizing multiple access points (APs), STAR-RISs, and NOMA is proposed for indoor communication.
  arXiv  Detail & Related papers  (2024-06-19T07:17:04Z)
• Joint User Association, Interference Cancellation and Power Control for Multi-IRS Assisted UAV Communications [80.35959154762381]
  Intelligent reflecting surface (IRS)-assisted unmanned aerial vehicle (UAV) communications are expected to alleviate the load of ground base stations in a cost-effective way. Existing studies mainly focus on the deployment and resource allocation of a single IRS instead of multiple IRSs. We propose a new optimization algorithm for joint IRS-user association, trajectory optimization of UAVs, successive interference cancellation (SIC) decoding order scheduling and power allocation.
  arXiv  Detail & Related papers  (2023-12-08T01:57:10Z)
• Lyapunov-Driven Deep Reinforcement Learning for Edge Inference Empowered by Reconfigurable Intelligent Surfaces [30.1512069754603]
  We propose a novel algorithm for energy-efficient, low-latency, accurate inference at the wireless edge. We consider a scenario where new data are continuously generated/collected by a set of devices and are handled through a dynamic queueing system.
  arXiv  Detail & Related papers  (2023-05-18T12:46:42Z)
• Learning OFDM Waveforms with PAPR and ACLR Constraints [15.423422040627331]
  We propose a learning-based method to design OFDM-based waveforms that satisfy selected constraints while maximizing an achievable information rate. We show that the end-to-end system is able to satisfy target PAPR and ACLR constraints and allows significant throughput gains.
  arXiv  Detail & Related papers  (2021-10-21T08:58:59Z)
• Path Design and Resource Management for NOMA enhanced Indoor Intelligent Robots [58.980293789967575]
  A communication enabled indoor intelligent robots (IRs) service framework is proposed. Lego modeling method is proposed, which can deterministically describe the indoor layout and channel state. The investigated radio map is invoked as a virtual environment to train the reinforcement learning agent.
  arXiv  Detail & Related papers  (2020-11-23T21:45:01Z)
• Optimization-driven Machine Learning for Intelligent Reflecting Surfaces Assisted Wireless Networks [82.33619654835348]
  Intelligent surface (IRS) has been employed to reshape the wireless channels by controlling individual scattering elements' phase shifts. Due to the large size of scattering elements, the passive beamforming is typically challenged by the high computational complexity. In this article, we focus on machine learning (ML) approaches for performance in IRS-assisted wireless networks.
  arXiv  Detail & Related papers  (2020-08-29T08:39:43Z)
• A Reinforcement Learning based approach for Multi-target Detection in Massive MIMO radar [12.982044791524494]
  This paper considers the problem of multi-target detection for massive multiple input multiple output (MMIMO) cognitive radar (CR) We propose a reinforcement learning (RL) based algorithm for cognitive multi-target detection in the presence of unknown disturbance statistics. Numerical simulations are performed to assess the performance of the proposed RL-based algorithm in both stationary and dynamic environments.
  arXiv  Detail & Related papers  (2020-05-10T16:29:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.