A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing

• URL: http://arxiv.org/abs/2110.02736v1
• Date: Tue, 5 Oct 2021 03:00:33 GMT
• Title: A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing
• Authors: Akash Doshi, Srinivas Yerramalli, Lorenzo Ferrari, Taesang Yoo, Jeffrey G. Andrews
• Abstract summary: We consider decentralized contention-based medium access for base stations operating on unlicensed shared spectrum. We introduce a two-stage Markov decision process in each time slot that uses information from spectrum sensing and reception quality to make a medium access decision. Our formulation provides decentralized inference, online adaptability and also caters to partial observability of the environment.
• Score: 31.640828282666245
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The increasing number of wireless devices operating in unlicensed spectrum motivates the development of intelligent adaptive approaches to spectrum access. We consider decentralized contention-based medium access for base stations (BSs) operating on unlicensed shared spectrum, where each BS autonomously decides whether or not to transmit on a given resource. The contention decision attempts to maximize not its own downlink throughput, but rather a network-wide objective. We formulate this problem as a decentralized partially observable Markov decision process with a novel reward structure that provides long term proportional fairness in terms of throughput. We then introduce a two-stage Markov decision process in each time slot that uses information from spectrum sensing and reception quality to make a medium access decision. Finally, we incorporate these features into a distributed reinforcement learning framework for contention-based spectrum access. Our formulation provides decentralized inference, online adaptability and also caters to partial observability of the environment through recurrent Q-learning. Empirically, we find its maximization of the proportional fairness metric to be competitive with a genie-aided adaptive energy detection threshold, while being robust to channel fading and small contention windows.

Related papers

• Artificial Intelligence Empowered Multiple Access for Ultra Reliable and Low Latency THz Wireless Networks [76.89730672544216]
  Terahertz (THz) wireless networks are expected to catalyze the beyond fifth generation (B5G) era. To satisfy the ultra-reliability and low-latency demands of several B5G applications, novel mobility management approaches are required. This article presents a holistic MAC layer approach that enables intelligent user association and resource allocation, as well as flexible and adaptive mobility management.
  arXiv  Detail & Related papers  (2022-08-17T03:00:24Z)
• Learning Resilient Radio Resource Management Policies with Graph Neural Networks [124.89036526192268]
  We formulate a resilient radio resource management problem with per-user minimum-capacity constraints. We show that we can parameterize the user selection and power control policies using a finite set of parameters. Thanks to such adaptation, our proposed method achieves a superior tradeoff between the average rate and the 5th percentile rate.
  arXiv  Detail & Related papers  (2022-03-07T19:40:39Z)
• A Q-Learning-based Approach for Distributed Beam Scheduling in mmWave Networks [18.22250038264899]
  We consider the problem of distributed downlink beam scheduling and power allocation for millimeter-Wave (mmWave) cellular networks. Multiple base stations belonging to different service operators share the same unlicensed spectrum with no central coordination or cooperation among them. We propose a distributed scheduling approach to power allocation and adaptation for efficient interference management over the shared spectrum by modeling each BS as an independent Q-learning agent.
  arXiv  Detail & Related papers  (2021-10-17T02:58:13Z)
• Distributed Proximal Policy Optimization for Contention-Based Spectrum Access [40.99534735484468]
  We develop a novel distributed implementation of a policy gradient method known as Proximal Policy Optimization. In each time slot, a base station uses information from spectrum sensing and reception quality to autonomously decide whether or not to transmit on a given resource. We find the proportional fairness reward accumulated by the policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold.
  arXiv  Detail & Related papers  (2021-10-07T00:54:03Z)
• Distributed Deep Reinforcement Learning for Adaptive Medium Access and Modulation in Shared Spectrum [42.54329256803276]
  We study decentralized contention-based medium access for base stations operating on unlicensed shared spectrum. We devise a learning-based algorithm for both contention and adaptive modulation that attempts to maximize a network-wide downlink throughput objective. Empirically, we find the (proportional fairness) reward accumulated by the policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold.
  arXiv  Detail & Related papers  (2021-09-24T03:33:45Z)
• Optimal Power Allocation for Rate Splitting Communications with Deep Reinforcement Learning [61.91604046990993]
  This letter introduces a novel framework to optimize the power allocation for users in a Rate Splitting Multiple Access network. In the network, messages intended for users are split into different parts that are a single common part and respective private parts.
  arXiv  Detail & Related papers  (2021-07-01T06:32:49Z)
• Federated Dynamic Spectrum Access [29.302039892247787]
  We introduce a Federated Learning (FL) based framework for the task of Dynamic Spectrum Access (DSA) FL is a distributive machine learning framework that can reserve the privacy of network terminals under heterogeneous data distributions.
  arXiv  Detail & Related papers  (2021-06-28T20:49:41Z)
• Cooperative Multi-Agent Reinforcement Learning Based Distributed Dynamic Spectrum Access in Cognitive Radio Networks [46.723006378363785]
  Dynamic spectrum access (DSA) is a promising paradigm to remedy the problem of inefficient spectrum utilization. In this paper, we investigate the distributed DSA problem for multi-user in a typical cognitive radio network. We employ the deep recurrent Q-network (DRQN) to address the partial observability of the state for each cognitive user.
  arXiv  Detail & Related papers  (2021-06-17T06:52:21Z)
• Decentralized Learning for Channel Allocation in IoT Networks over Unlicensed Bandwidth as a Contextual Multi-player Multi-armed Bandit Game [134.88020946767404]
  We study a decentralized channel allocation problem in an ad-hoc Internet of Things network underlaying on the spectrum licensed to a primary cellular network. Our study maps this problem into a contextual multi-player, multi-armed bandit game, and proposes a purely decentralized, three-stage policy learning algorithm through trial-and-error.
  arXiv  Detail & Related papers  (2020-03-30T10:05:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.