Generalization of Deep Reinforcement Learning for Jammer-Resilient Frequency and Power Allocation

• URL: http://arxiv.org/abs/2302.02250v2
• Date: Sat, 6 May 2023 19:03:14 GMT
• Title: Generalization of Deep Reinforcement Learning for Jammer-Resilient Frequency and Power Allocation
• Authors: Swatantra Kafle, Jithin Jagannath, Zackary Kane, Noor Biswas, Prem Sagar Vasanth Kumar, Anu Jagannath
• Abstract summary: We tackle the problem of joint frequency and power allocation while emphasizing the generalization capability of a deep reinforcement learning model. We show the improved training and inference performance of the proposed methods when tested on previously unseen simulated wireless networks. The end-to-end solution was implemented on the embedded software-defined radio and validated using over-the-air evaluation.
• Score: 4.436632973105495
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We tackle the problem of joint frequency and power allocation while emphasizing the generalization capability of a deep reinforcement learning model. Most of the existing methods solve reinforcement learning-based wireless problems for a specific pre-determined wireless network scenario. The performance of a trained agent tends to be very specific to the network and deteriorates when used in a different network operating scenario (e.g., different in size, neighborhood, and mobility, among others). We demonstrate our approach to enhance training to enable a higher generalization capability during inference of the deployed model in a distributed multi-agent setting in a hostile jamming environment. With all these, we show the improved training and inference performance of the proposed methods when tested on previously unseen simulated wireless networks of different sizes and architectures. More importantly, to prove practical impact, the end-to-end solution was implemented on the embedded software-defined radio and validated using over-the-air evaluation.

Related papers

• Multi-Agent Reinforcement Learning for Power Control in Wireless Networks via Adaptive Graphs [1.1861167902268832]
  Multi-agent deep reinforcement learning (MADRL) has emerged as a promising method to address a wide range of complex optimization problems like power control. We present the use of graphs as communication-inducing structures among distributed agents as an effective means to mitigate these challenges.
  arXiv  Detail & Related papers  (2023-11-27T14:25:40Z)
• Deep Reinforcement Learning for mmWave Initial Beam Alignment [6.240268911509346]
  We investigate the applicability of deep reinforcement learning algorithms to the adaptive initial access beam alignment problem for mmWave communications. Deep reinforcement learning has the potential to address a new and wider range of applications. We show that, although the chosen off-the-shelf deep reinforcement learning agent fails to perform well when trained on realistic problem sizes, introducing action space shaping in the form of beamforming modules vastly improves the performance.
  arXiv  Detail & Related papers  (2023-02-17T16:10:42Z)
• Multi-agent Reinforcement Learning with Graph Q-Networks for Antenna Tuning [60.94661435297309]
  The scale of mobile networks makes it challenging to optimize antenna parameters using manual intervention or hand-engineered strategies. We propose a new multi-agent reinforcement learning algorithm to optimize mobile network configurations globally. We empirically demonstrate the performance of the algorithm on an antenna tilt tuning problem and a joint tilt and power control problem in a simulated environment.
  arXiv  Detail & Related papers  (2023-01-20T17:06:34Z)
• Learning Dynamics and Generalization in Reinforcement Learning [59.530058000689884]
  We show theoretically that temporal difference learning encourages agents to fit non-smooth components of the value function early in training. We show that neural networks trained using temporal difference algorithms on dense reward tasks exhibit weaker generalization between states than randomly networks and gradient networks trained with policy methods.
  arXiv  Detail & Related papers  (2022-06-05T08:49:16Z)
• Offline Contextual Bandits for Wireless Network Optimization [107.24086150482843]
  In this paper, we investigate how to learn policies that can automatically adjust the configuration parameters of every cell in the network in response to the changes in the user demand. Our solution combines existent methods for offline learning and adapts them in a principled way to overcome crucial challenges arising in this context.
  arXiv  Detail & Related papers  (2021-11-11T11:31:20Z)
• In-Network Learning: Distributed Training and Inference in Networks [10.635097939284753]
  We develop a learning algorithm and an architecture that make use of multiple data streams and processing units. In particular, the analysis reveals how inference propagates and fuses across a network.
  arXiv  Detail & Related papers  (2021-07-07T18:35:08Z)
• All at Once Network Quantization via Collaborative Knowledge Transfer [56.95849086170461]
  We develop a novel collaborative knowledge transfer approach for efficiently training the all-at-once quantization network. Specifically, we propose an adaptive selection strategy to choose a high-precision enquoteteacher for transferring knowledge to the low-precision student. To effectively transfer knowledge, we develop a dynamic block swapping method by randomly replacing the blocks in the lower-precision student network with the corresponding blocks in the higher-precision teacher network.
  arXiv  Detail & Related papers  (2021-03-02T03:09:03Z)
• Reinforcement Learning for Datacenter Congestion Control [50.225885814524304]
  Successful congestion control algorithms can dramatically improve latency and overall network throughput. Until today, no such learning-based algorithms have shown practical potential in this domain. We devise an RL-based algorithm with the aim of generalizing to different configurations of real-world datacenter networks. We show that this scheme outperforms alternative popular RL approaches, and generalizes to scenarios that were not seen during training.
  arXiv  Detail & Related papers  (2021-02-18T13:49:28Z)
• Deep Actor-Critic Learning for Distributed Power Control in Wireless Mobile Networks [5.930707872313038]
  Deep reinforcement learning offers a model-free alternative to supervised deep learning and classical optimization. We present a distributively executed continuous power control algorithm with the help of deep actor-critic learning. We integrate the proposed power control algorithm to a time-slotted system where devices are mobile and channel conditions change rapidly.
  arXiv  Detail & Related papers  (2020-09-14T18:29:12Z)
• Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning [90.93035276307239]
  We propose an information theoretic regularization objective and an annealing-based optimization method to achieve better generalization ability in RL agents. We demonstrate the extreme generalization benefits of our approach in different domains ranging from maze navigation to robotic tasks. This work provides a principled way to improve generalization in RL by gradually removing information that is redundant for task-solving.
  arXiv  Detail & Related papers  (2020-08-03T02:24:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.