Robust Reinforcement Learning over Wireless Networks with Homomorphic State Representations

• URL: http://arxiv.org/abs/2508.07722v1
• Date: Mon, 11 Aug 2025 07:50:25 GMT
• Title: Robust Reinforcement Learning over Wireless Networks with Homomorphic State Representations
• Authors: Pietro Talli, Federico Mason, Federico Chiariotti, Andrea Zanella,
• Abstract summary: We address the problem of training Reinforcement Learning (RL) agents over communication networks.<n>We propose a novel architecture, named Homomorphic Robust Remote Reinforcement Learning (HR3L)<n>HR3L does not require the exchange of gradient information across the wireless channel, allowing for quicker training and a lower communication overhead than state-of-the-art solutions.
• Score: 15.079887992932692
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we address the problem of training Reinforcement Learning (RL) agents over communication networks. The RL paradigm requires the agent to instantaneously perceive the state evolution to infer the effects of its actions on the environment. This is impossible if the agent receives state updates over lossy or delayed wireless systems and thus operates with partial and intermittent information. In recent years, numerous frameworks have been proposed to manage RL with imperfect feedback; however, they often offer specific solutions with a substantial computational burden. To address these limits, we propose a novel architecture, named Homomorphic Robust Remote Reinforcement Learning (HR3L), that enables the training of remote RL agents exchanging observations across a non-ideal wireless channel. HR3L considers two units: the transmitter, which encodes meaningful representations of the environment, and the receiver, which decodes these messages and performs actions to maximize a reward signal. Importantly, HR3L does not require the exchange of gradient information across the wireless channel, allowing for quicker training and a lower communication overhead than state-of-the-art solutions. Experimental results demonstrate that HR3L significantly outperforms baseline methods in terms of sample efficiency and adapts to different communication scenarios, including packet losses, delayed transmissions, and capacity limitations.

Related papers

• Communication-Efficient Multi-Modal Edge Inference via Uncertainty-Aware Distributed Learning [60.650628083185616]
  We propose a three-stage communication-aware distributed learning framework to improve training and inference efficiency.<n>In StageI, devices perform local multi-modal self-supervised learning to obtain shared and modality-specific encoders without device--server exchange.<n>StageII, distributed fine-tuning with centralized evidential fusion calibrates per-modality uncertainty and reliably aggregates features distorted by noise or channel fading.<n>StageIII, an uncertainty-guided feedback mechanism selectively requests additional features for uncertain samples, optimizing the communication--accuracy tradeoff in the distributed setting.
  arXiv  Detail & Related papers  (2026-01-21T12:38:02Z)
• Large Language Model (LLM)-enabled Reinforcement Learning for Wireless Network Optimization [79.27012080083603]
  Large language models (LLMs) offer promising tools to enhance reinforcement learning in wireless networks.<n>We propose an LLM-assisted state representation and semantic extraction to enhance the multi-agent reinforcement learning framework.
  arXiv  Detail & Related papers  (2026-01-15T01:42:39Z)
• Efficient Beam Selection for ISAC in Cell-Free Massive MIMO via Digital Twin-Assisted Deep Reinforcement Learning [37.540612510652174]
  We derive the distribution of joint target detection probabilities across multiple receiving APs under false alarm rate constraints.<n>We then formulate the beam selection procedure as a Markov decision process (MDP)<n>To eliminate the high costs and associated risks of real-time agent-environment interactions, we propose a novel digital twin (DT)-assisted offline DRL approach.
  arXiv  Detail & Related papers  (2025-06-23T12:17:57Z)
• Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models [2.702550149035333]
  We propose Diffusion-based Deep Reinforcement Learning (D2RL) to overcome fundamental DRL training bottlenecks for wireless networks.<n>D2RL achieves faster convergence and reduced computational costs over conventional DRL methods for resource allocation in wireless communications.<n>This work underscores the transformative potential of GDMs in overcoming fundamental DRL training bottlenecks for wireless networks.
  arXiv  Detail & Related papers  (2025-02-11T03:09:45Z)
• A Memory-Based Reinforcement Learning Approach to Integrated Sensing and Communication [52.40430937325323]
  We consider a point-to-point integrated sensing and communication (ISAC) system, where a transmitter conveys a message to a receiver over a channel with memory.<n>We formulate the capacity-distortion tradeoff for the ISAC problem when sensing is performed in an online fashion.
  arXiv  Detail & Related papers  (2024-12-02T03:30:50Z)
• Offline Reinforcement Learning and Sequence Modeling for Downlink Link Adaptation [3.687363450234871]
  Link adaptation (LA) is an essential function in modern wireless communication systems.<n>LA dynamically adjusts the transmission rate of a communication link to match time- and frequency-varying radio link conditions.<n>Recent research has introduced online reinforcement learning approaches as an alternative to the more commonly used rule-based algorithms.
  arXiv  Detail & Related papers  (2024-10-30T14:01:31Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [94.2860766709971]
  We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents.<n>Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
  arXiv  Detail & Related papers  (2024-04-04T06:24:11Z)
• Effective Communication with Dynamic Feature Compression [25.150266946722]
  We study a prototypal system in which an observer must communicate its sensory data to a robot controlling a task. We consider an ensemble Vector Quantized Variational Autoencoder (VQ-VAE) encoding, and train a Deep Reinforcement Learning (DRL) agent to dynamically adapt the quantization level. We tested the proposed approach on the well-known CartPole reference control problem, obtaining a significant performance increase.
  arXiv  Detail & Related papers  (2024-01-29T15:35:05Z)
• Semantic and Effective Communication for Remote Control Tasks with Dynamic Feature Compression [23.36744348465991]
  Coordination of robotic swarms and the remote wireless control of industrial systems are among the major use cases for 5G and beyond systems. In this work, we consider a prototypal system in which an observer must communicate its sensory data to an actor controlling a task. We propose an ensemble Vector Quantized Variational Autoencoder (VQ-VAE) encoding, and train a Deep Reinforcement Learning (DRL) agent to dynamically adapt the quantization level.
  arXiv  Detail & Related papers  (2023-01-14T11:43:56Z)
• Model-based Deep Learning Receiver Design for Rate-Splitting Multiple Access [65.21117658030235]
  This work proposes a novel design for a practical RSMA receiver based on model-based deep learning (MBDL) methods. The MBDL receiver is evaluated in terms of uncoded Symbol Error Rate (SER), throughput performance through Link-Level Simulations (LLS) and average training overhead. Results reveal that the MBDL outperforms by a significant margin the SIC receiver with imperfect CSIR.
  arXiv  Detail & Related papers  (2022-05-02T12:23:55Z)
• Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks [82.02891936174221]
  Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach. In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
  arXiv  Detail & Related papers  (2021-11-23T18:24:47Z)
• Federated Learning over Wireless IoT Networks with Optimized Communication and Resources [98.18365881575805]
  Federated learning (FL) as a paradigm of collaborative learning techniques has obtained increasing research attention. It is of interest to investigate fast responding and accurate FL schemes over wireless systems. We show that the proposed communication-efficient federated learning framework converges at a strong linear rate.
  arXiv  Detail & Related papers  (2021-10-22T13:25:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.