Cooperative Multi-Agent Reinforcement Learning Based Distributed Dynamic Spectrum Access in Cognitive Radio Networks

• URL: http://arxiv.org/abs/2106.09274v1
• Date: Thu, 17 Jun 2021 06:52:21 GMT
• Title: Cooperative Multi-Agent Reinforcement Learning Based Distributed Dynamic Spectrum Access in Cognitive Radio Networks
• Authors: Xiang Tan, Li Zhou, Haijun Wang, Yuli Sun, Haitao Zhao, Boon-Chong Seet, Jibo Wei and Victor C.M. Leung
• Abstract summary: Dynamic spectrum access (DSA) is a promising paradigm to remedy the problem of inefficient spectrum utilization. In this paper, we investigate the distributed DSA problem for multi-user in a typical cognitive radio network. We employ the deep recurrent Q-network (DRQN) to address the partial observability of the state for each cognitive user.
• Score: 46.723006378363785
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the development of the 5G and Internet of Things, amounts of wireless devices need to share the limited spectrum resources. Dynamic spectrum access (DSA) is a promising paradigm to remedy the problem of inefficient spectrum utilization brought upon by the historical command-and-control approach to spectrum allocation. In this paper, we investigate the distributed DSA problem for multi-user in a typical multi-channel cognitive radio network. The problem is formulated as a decentralized partially observable Markov decision process (Dec-POMDP), and we proposed a centralized off-line training and distributed on-line execution framework based on cooperative multi-agent reinforcement learning (MARL). We employ the deep recurrent Q-network (DRQN) to address the partial observability of the state for each cognitive user. The ultimate goal is to learn a cooperative strategy which maximizes the sum throughput of cognitive radio network in distributed fashion without coordination information exchange between cognitive users. Finally, we validate the proposed algorithm in various settings through extensive experiments. From the simulation results, we can observe that the proposed algorithm can converge fast and achieve almost the optimal performance.

Related papers

• Decentralized multi-agent reinforcement learning algorithm using a cluster-synchronized laser network [1.124958340749622]
  We propose a photonic-based decision-making algorithm to address the competitive multi-armed bandit problem. Our numerical simulations demonstrate that chaotic oscillations and cluster synchronization of optically coupled lasers, along with our proposed decentralized coupling adjustment, efficiently balance exploration and exploitation.
  arXiv  Detail & Related papers  (2024-07-12T09:38:47Z)
• Multi-Agent Hybrid SAC for Joint SS-DSA in CRNs [17.162697767466085]
  Opportunistic spectrum access has the potential to increase the efficiency of spectrum utilization in cognitive radio networks (CRNs) In CRNs, both spectrum sensing and resource allocation (SSRA) are critical to maximizing system throughput. We develop a novel multi-agent implementation of hybrid soft actor critic, MHSAC.
  arXiv  Detail & Related papers  (2024-04-22T16:30:03Z)
• Decentralized Learning over Wireless Networks: The Effect of Broadcast with Random Access [56.91063444859008]
  We investigate the impact of broadcast transmission and probabilistic random access policy on the convergence performance of D-SGD. Our results demonstrate that optimizing the access probability to maximize the expected number of successful links is a highly effective strategy for accelerating the system convergence.
  arXiv  Detail & Related papers  (2023-05-12T10:32:26Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks [82.02891936174221]
  Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach. In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
  arXiv  Detail & Related papers  (2021-11-23T18:24:47Z)
• A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing [31.640828282666245]
  We consider decentralized contention-based medium access for base stations operating on unlicensed shared spectrum. We introduce a two-stage Markov decision process in each time slot that uses information from spectrum sensing and reception quality to make a medium access decision. Our formulation provides decentralized inference, online adaptability and also caters to partial observability of the environment.
  arXiv  Detail & Related papers  (2021-10-05T03:00:33Z)
• Distributed Deep Reinforcement Learning for Adaptive Medium Access and Modulation in Shared Spectrum [42.54329256803276]
  We study decentralized contention-based medium access for base stations operating on unlicensed shared spectrum. We devise a learning-based algorithm for both contention and adaptive modulation that attempts to maximize a network-wide downlink throughput objective. Empirically, we find the (proportional fairness) reward accumulated by the policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold.
  arXiv  Detail & Related papers  (2021-09-24T03:33:45Z)
• Distributed Deep Reinforcement Learning for Collaborative Spectrum Sharing [29.23509739013885]
  We discuss the problem of distributed spectrum collaboration without central management under general unknown channels. We combine game-theoretic insights with deep Q-learning to provide a novelally optimal solution to the spectrum collaboration problem.
  arXiv  Detail & Related papers  (2021-04-06T04:33:06Z)
• Deep Multi-Task Learning for Cooperative NOMA: System Design and Principles [52.79089414630366]
  We develop a novel deep cooperative NOMA scheme, drawing upon the recent advances in deep learning (DL) We develop a novel hybrid-cascaded deep neural network (DNN) architecture such that the entire system can be optimized in a holistic manner.
  arXiv  Detail & Related papers  (2020-07-27T12:38:37Z)
• Modality Compensation Network: Cross-Modal Adaptation for Action Recognition [77.24983234113957]
  We propose a Modality Compensation Network (MCN) to explore the relationships of different modalities. Our model bridges data from source and auxiliary modalities by a modality adaptation block to achieve adaptive representation learning. Experimental results reveal that MCN outperforms state-of-the-art approaches on four widely-used action recognition benchmarks.
  arXiv  Detail & Related papers  (2020-01-31T04:51:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.