Distributed Deep Reinforcement Learning for Adaptive Medium Access and Modulation in Shared Spectrum

• URL: http://arxiv.org/abs/2109.11723v1
• Date: Fri, 24 Sep 2021 03:33:45 GMT
• Title: Distributed Deep Reinforcement Learning for Adaptive Medium Access and Modulation in Shared Spectrum
• Authors: Akash Doshi and Jeffrey G. Andrews
• Abstract summary: We study decentralized contention-based medium access for base stations operating on unlicensed shared spectrum. We devise a learning-based algorithm for both contention and adaptive modulation that attempts to maximize a network-wide downlink throughput objective. Empirically, we find the (proportional fairness) reward accumulated by the policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold.
• Score: 42.54329256803276
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spectrum scarcity has led to growth in the use of unlicensed spectrum for cellular systems. This motivates intelligent adaptive approaches to spectrum access for both WiFi and 5G that improve upon traditional carrier sensing and listen-before-talk methods. We study decentralized contention-based medium access for base stations (BSs) of a single Radio Access Technology (RAT) operating on unlicensed shared spectrum. We devise a learning-based algorithm for both contention and adaptive modulation that attempts to maximize a network-wide downlink throughput objective. We formulate and develop novel distributed implementations of two deep reinforcement learning approaches - Deep Q Networks and Proximal Policy Optimization - modelled on a two stage Markov decision process. Empirically, we find the (proportional fairness) reward accumulated by the policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold. Our approaches are further validated by improved sum and peak throughput. The scalability of our approach to large networks is demonstrated via an improved cumulative reward earned on both indoor and outdoor layouts with a large number of BSs.

Related papers

• Dynamic Spectrum Access for Ambient Backscatter Communication-assisted D2D Systems with Quantum Reinforcement Learning [68.63990729719369]
  The wireless spectrum is becoming scarce, resulting in low spectral efficiency for D2D communications. This paper aims to integrate the ambient backscatter communication technology into D2D devices to allow them to backscatter ambient RF signals. We develop a novel quantum reinforcement learning (RL) algorithm that can achieve a faster convergence rate with fewer training parameters.
  arXiv  Detail & Related papers  (2024-10-23T15:36:43Z)
• Label-free Deep Learning Driven Secure Access Selection in Space-Air-Ground Integrated Networks [26.225658457052834]
  In space-air-ground integrated networks (SAGIN), the inherent openness and extensive broadcast coverage expose these networks to significant eavesdropping threats. It is challenging to conduct a secrecy-oriented access strategy due to both heterogeneous resources and different eavesdropping models. We propose a Q-network approximation based deep learning approach for selecting the optimal access strategy for maximizing the sum secrecy rate.
  arXiv  Detail & Related papers  (2023-08-28T06:48:06Z)
• An Unsupervised Learning Approach for Spectrum Allocation in Terahertz Communication Systems [31.263991262752498]
  We propose a new spectrum allocation strategy, aided by unsupervised learning, for terahertz communication systems. We first formulate an optimization problem to determine the optimal sub-band bandwidth and transmit power. We then propose the unsupervised learning-based approach to obtaining the near-optimal solution to this problem.
  arXiv  Detail & Related papers  (2022-08-07T02:14:13Z)
• Distributed Proximal Policy Optimization for Contention-Based Spectrum Access [40.99534735484468]
  We develop a novel distributed implementation of a policy gradient method known as Proximal Policy Optimization. In each time slot, a base station uses information from spectrum sensing and reception quality to autonomously decide whether or not to transmit on a given resource. We find the proportional fairness reward accumulated by the policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold.
  arXiv  Detail & Related papers  (2021-10-07T00:54:03Z)
• A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing [31.640828282666245]
  We consider decentralized contention-based medium access for base stations operating on unlicensed shared spectrum. We introduce a two-stage Markov decision process in each time slot that uses information from spectrum sensing and reception quality to make a medium access decision. Our formulation provides decentralized inference, online adaptability and also caters to partial observability of the environment.
  arXiv  Detail & Related papers  (2021-10-05T03:00:33Z)
• Cooperative Multi-Agent Reinforcement Learning Based Distributed Dynamic Spectrum Access in Cognitive Radio Networks [46.723006378363785]
  Dynamic spectrum access (DSA) is a promising paradigm to remedy the problem of inefficient spectrum utilization. In this paper, we investigate the distributed DSA problem for multi-user in a typical cognitive radio network. We employ the deep recurrent Q-network (DRQN) to address the partial observability of the state for each cognitive user.
  arXiv  Detail & Related papers  (2021-06-17T06:52:21Z)
• Meta-Reinforcement Learning for Trajectory Design in Wireless UAV Networks [151.65541208130995]
  A drone base station (DBS) is dispatched to provide uplink connectivity to ground users whose demand is dynamic and unpredictable. In this case, the DBS's trajectory must be adaptively adjusted to satisfy the dynamic user access requests. A meta-learning algorithm is proposed in order to adapt the DBS's trajectory when it encounters novel environments.
  arXiv  Detail & Related papers  (2020-05-25T20:43:59Z)
• Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications [54.610318402371185]
  Intelligent reflecting surface (IRS) is a promising technology to assist downlink information transmissions from a multi-antenna access point (AP) to a receiver. We minimize the AP's transmit power by a joint optimization of the AP's active beamforming and the IRS's passive beamforming. We propose a deep reinforcement learning (DRL) approach that can adapt the beamforming strategies from past experiences.
  arXiv  Detail & Related papers  (2020-05-25T01:42:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.