Causality-Driven Reinforcement Learning for Joint Communication and Sensing

• URL: http://arxiv.org/abs/2409.15329v1
• Date: Sat, 7 Sep 2024 07:15:57 GMT
• Title: Causality-Driven Reinforcement Learning for Joint Communication and Sensing
• Authors: Anik Roy, Serene Banerjee, Jishnu Sadasivan, Arnab Sarkar, Soumyajit Dey,
• Abstract summary: We propose a causally-aware RL agent which can intervene and discover causal relationships for mMIMO-based JCAS environments. We use a state dependent action dimension selection strategy to realize causal discovery for RL-based JCAS.
• Score: 4.165335263540595
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The next-generation wireless network, 6G and beyond, envisions to integrate communication and sensing to overcome interference, improve spectrum efficiency, and reduce hardware and power consumption. Massive Multiple-Input Multiple Output (mMIMO)-based Joint Communication and Sensing (JCAS) systems realize this integration for 6G applications such as autonomous driving, as it requires accurate environmental sensing and time-critical communication with neighboring vehicles. Reinforcement Learning (RL) is used for mMIMO antenna beamforming in the existing literature. However, the huge search space for actions associated with antenna beamforming causes the learning process for the RL agent to be inefficient due to high beam training overhead. The learning process does not consider the causal relationship between action space and the reward, and gives all actions equal importance. In this work, we explore a causally-aware RL agent which can intervene and discover causal relationships for mMIMO-based JCAS environments, during the training phase. We use a state dependent action dimension selection strategy to realize causal discovery for RL-based JCAS. Evaluation of the causally-aware RL framework in different JCAS scenarios shows the benefit of our proposed framework over baseline methods in terms of the beamforming gain.

Related papers

• iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvement [5.078593258867346]
  We introduce a novel integration framework that combines a large language model (LLM) with reinforcement learning (RL) Our approach reduces the average waiting time by $17.5%$ in degraded communication conditions as compared to traditional RL methods.
  arXiv  Detail & Related papers  (2024-07-08T15:22:49Z)
• Effective Communication with Dynamic Feature Compression [25.150266946722]
  We study a prototypal system in which an observer must communicate its sensory data to a robot controlling a task. We consider an ensemble Vector Quantized Variational Autoencoder (VQ-VAE) encoding, and train a Deep Reinforcement Learning (DRL) agent to dynamically adapt the quantization level. We tested the proposed approach on the well-known CartPole reference control problem, obtaining a significant performance increase.
  arXiv  Detail & Related papers  (2024-01-29T15:35:05Z)
• Effective Intrusion Detection in Heterogeneous Internet-of-Things Networks via Ensemble Knowledge Distillation-based Federated Learning [52.6706505729803]
  We introduce Federated Learning (FL) to collaboratively train a decentralized shared model of Intrusion Detection Systems (IDS) FLEKD enables a more flexible aggregation method than conventional model fusion techniques. Experiment results show that the proposed approach outperforms local training and traditional FL in terms of both speed and performance.
  arXiv  Detail & Related papers  (2024-01-22T14:16:37Z)
• Multiagent Reinforcement Learning with an Attention Mechanism for Improving Energy Efficiency in LoRa Networks [52.96907334080273]
  As the network scale increases, the energy efficiency of LoRa networks decreases sharply due to severe packet collisions. We propose a transmission parameter allocation algorithm based on multiagent reinforcement learning (MALoRa) Simulation results demonstrate that MALoRa significantly improves the system EE compared with baseline algorithms.
  arXiv  Detail & Related papers  (2023-09-16T11:37:23Z)
• Learning to Sail Dynamic Networks: The MARLIN Reinforcement Learning Framework for Congestion Control in Tactical Environments [53.08686495706487]
  This paper proposes an RL framework that leverages an accurate and parallelizable emulation environment to reenact the conditions of a tactical network. We evaluate our RL learning framework by training a MARLIN agent in conditions replicating a bottleneck link transition between a Satellite Communication (SATCOM) and an UHF Wide Band (UHF) radio link.
  arXiv  Detail & Related papers  (2023-06-27T16:15:15Z)
• Semantic and Effective Communication for Remote Control Tasks with Dynamic Feature Compression [23.36744348465991]
  Coordination of robotic swarms and the remote wireless control of industrial systems are among the major use cases for 5G and beyond systems. In this work, we consider a prototypal system in which an observer must communicate its sensory data to an actor controlling a task. We propose an ensemble Vector Quantized Variational Autoencoder (VQ-VAE) encoding, and train a Deep Reinforcement Learning (DRL) agent to dynamically adapt the quantization level.
  arXiv  Detail & Related papers  (2023-01-14T11:43:56Z)
• Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks [82.02891936174221]
  Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach. In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
  arXiv  Detail & Related papers  (2021-11-23T18:24:47Z)
• Federated Learning over Wireless IoT Networks with Optimized Communication and Resources [98.18365881575805]
  Federated learning (FL) as a paradigm of collaborative learning techniques has obtained increasing research attention. It is of interest to investigate fast responding and accurate FL schemes over wireless systems. We show that the proposed communication-efficient federated learning framework converges at a strong linear rate.
  arXiv  Detail & Related papers  (2021-10-22T13:25:57Z)
• Vehicular Cooperative Perception Through Action Branching and Federated Reinforcement Learning [101.64598586454571]
  A novel framework is proposed to allow reinforcement learning-based vehicular association, resource block (RB) allocation, and content selection of cooperative perception messages (CPMs) A federated RL approach is introduced in order to speed up the training process across vehicles. Results show that federated RL improves the training process, where better policies can be achieved within the same amount of time compared to the non-federated approach.
  arXiv  Detail & Related papers  (2020-12-07T02:09:15Z)
• Hierarchical Reinforcement Learning for Relay Selection and Power Optimization in Two-Hop Cooperative Relay Network [7.5377621697101205]
  We study the outage probability minimizing problem subjected to a total transmission power constraint in a two-hop cooperative relay network. We use reinforcement learning (RL) methods to learn strategies for relay selection and power allocation. We propose a hierarchical reinforcement learning (HRL) framework and training algorithm.
  arXiv  Detail & Related papers  (2020-11-10T04:47:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.