Learning to Focus: CSI-Free Hierarchical MARL for Reconfigurable Reflectors

• URL: http://arxiv.org/abs/2604.05165v2
• Date: Mon, 13 Apr 2026 15:27:38 GMT
• Title: Learning to Focus: CSI-Free Hierarchical MARL for Reconfigurable Reflectors
• Abstract summary: Reconfigurable Intelligent Surfaces (RIS) has a potential to engineer smart radio environments for next-generation millimeter-wave (mmWave) networks.<n>We introduce a CSI-free" paradigm powered by a Hierarchical Multi-Agent Reinforcement Learning (HMARL) architecture to control mechanically reconfigurable surfaces.
• Score: 8.046649921554472
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reconfigurable Intelligent Surfaces (RIS) has a potential to engineer smart radio environments for next-generation millimeter-wave (mmWave) networks. However, the prohibitive computational overhead of Channel State Information (CSI) estimation and the dimensionality explosion inherent in centralized optimization severely hinder practical large-scale deployments. To overcome these bottlenecks, we introduce a ``CSI-free" paradigm powered by a Hierarchical Multi-Agent Reinforcement Learning (HMARL) architecture to control mechanically reconfigurable reflective surfaces. By substituting pilot-based channel estimation with accessible user localization data, our framework leverages spatial intelligence for macro-scale wave propagation management. The control problem is decomposed into a two-tier neural architecture: a high-level controller executes temporally extended, discrete user-to-reflector allocations, while low-level controllers autonomously optimize continuous focal points utilizing Multi-Agent Proximal Policy Optimization (MAPPO) under a Centralized Training with Decentralized Execution (CTDE) scheme. Comprehensive deterministic ray-tracing evaluations demonstrate that this hierarchical framework achieves massive RSSI improvements of up to 7.79 dB over centralized baselines. Furthermore, the system exhibits robust multi-user scalability and maintains highly resilient beam-focusing performance under practical sub-meter localization tracking errors. By eliminating CSI overhead while maintaining high-fidelity signal redirection, this work establishes a scalable and cost-effective blueprint for intelligent wireless environments.

Related papers

• Bypassing the CSI Bottleneck: MARL-Driven Spatial Control for Reflector Arrays [8.046649921554472]
  This paper presents a fully autonomous Multi-Agent Reinforcement Learning (MARL) framework to control mechanically adjustable metallic reflector arrays.<n>High-fidelity ray-tracing simulations in dynamic non-line-of-sight (NLOS) environments demonstrate that this multi-agent approach rapidly adapts to user mobility.<n>These results validate the efficacy of MARL-driven spatial abstractions as a scalable, highly practical pathway toward AI-empowered wireless networks.
  arXiv  Detail & Related papers  (2026-04-06T20:47:50Z)
• FRIEND: Federated Learning for Joint Optimization of multi-RIS Configuration and Eavesdropper Intelligent Detection in B5G Networks [3.809709581796032]
  We present a novel framework for detecting malicious users in RIS-enhanced cell-free mmWave networks using Federated Learning (FL)<n>We show that the integration of FL and multi-RIS coordination improves approximately 30% the achieved secrecy rate (SR) compared to baseline non-RIS-assisted methods.<n>This work establishes a distributed, privacy-preserving approach to physical layer eavesdropping detection tailored for next-generation IIoT deployments.
  arXiv  Detail & Related papers  (2026-03-11T17:02:40Z)
• Learning to Reflect: Hierarchical Multi-Agent Reinforcement Learning for CSI-Free mmWave Beam-Focusing [8.046649921554472]
  We introduce a "CSI-free" paradigm that substitutes pilot-based channel estimation with readily available user localization data.<n>The proposed architecture decomposes the control into two levels: a high-level controller for user-to-reflector allocation and decentralized low-level controllers for low-level focal point optimization.
  arXiv  Detail & Related papers  (2026-03-07T22:46:21Z)
• Generative MIMO Beam Map Construction for Location Recovery and Beam Tracking [67.65578956523403]
  This paper proposes a generative framework to recover location labels directly from sparse channel state information (CSI) measurements.<n>Instead of directly storing raw CSI, we learn a compact low-dimensional radio map embedding and leverage a generative model to reconstruct the high-dimensional CSI.<n> Numerical experiments demonstrate that the proposed model can improve localization accuracy by over 30% and achieve a 20% capacity gain in non-line-of-sight (NLOS) scenarios.
  arXiv  Detail & Related papers  (2025-11-21T07:25:49Z)
• CSIYOLO: An Intelligent CSI-based Scatter Sensing Framework for Integrated Sensing and Communication Systems [18.244924566345027]
  ISAC is regarded as a promising technology for next-generation communication systems, enabling simultaneous data transmission and target sensing.<n> scatter sensing plays a crucial role in exploiting the full potential of ISAC and supporting applications such as autonomous driving and low-altitude economy.<n>We propose CSIYOLO, a framework that performs scatter localization using estimated CSI from a single base station-user equipment pair.
  arXiv  Detail & Related papers  (2025-09-15T06:46:39Z)
• Green Learning for STAR-RIS mmWave Systems with Implicit CSI [53.03358325565645]
  Green learning (GL)-based precoding framework is proposed for simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS)-aided millimeter-wave (mmWave) broadcasting systems.<n>Motivated by the emphasis on environmental sustainability in future 6G networks, this work adopts a transmission framework for scenarios where multiple users share identical information, improving spectral efficiency and reducing redundant transmissions and power consumption.
  arXiv  Detail & Related papers  (2025-09-08T15:56:06Z)
• Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach [51.63921041249406]
  Non-orthogonal multiple access (NOMA) enables multiple users to share the same frequency band, and simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) deploying STAR-RIS indoors presents challenges in interference mitigation, power consumption, and real-time configuration. A novel network architecture utilizing multiple access points (APs), STAR-RISs, and NOMA is proposed for indoor communication.
  arXiv  Detail & Related papers  (2024-06-19T07:17:04Z)
• Sensor Control for Information Gain in Dynamic, Sparse and Partially Observed Environments [1.5402666674186938]
  We present an approach for autonomous sensor control for information gathering under partially observable, dynamic and sparsely sampled environments. We extend the Deep Anticipatory Network (DAN) Reinforcement Learning framework by (1) improving exploration in sparse, non-stationary environments using a novel information gain reward. We also extend this problem to situations in which sampling from the intended RF spectrum/field is limited and propose a model-based version of the original RL algorithm that fine-tunes the controller via a model that is iteratively improved from the limited field sampling.
  arXiv  Detail & Related papers  (2022-11-03T00:03:14Z)
• Artificial Intelligence Empowered Multiple Access for Ultra Reliable and Low Latency THz Wireless Networks [76.89730672544216]
  Terahertz (THz) wireless networks are expected to catalyze the beyond fifth generation (B5G) era. To satisfy the ultra-reliability and low-latency demands of several B5G applications, novel mobility management approaches are required. This article presents a holistic MAC layer approach that enables intelligent user association and resource allocation, as well as flexible and adaptive mobility management.
  arXiv  Detail & Related papers  (2022-08-17T03:00:24Z)
• Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces [56.35676570414731]
  The emerging technology of Reconfigurable Intelligent Surfaces (RISs) is provisioned as an enabler of smart wireless environments. RISs offer a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces.
  arXiv  Detail & Related papers  (2022-05-08T06:21:33Z)
• Model-based Deep Learning Receiver Design for Rate-Splitting Multiple Access [65.21117658030235]
  This work proposes a novel design for a practical RSMA receiver based on model-based deep learning (MBDL) methods. The MBDL receiver is evaluated in terms of uncoded Symbol Error Rate (SER), throughput performance through Link-Level Simulations (LLS) and average training overhead. Results reveal that the MBDL outperforms by a significant margin the SIC receiver with imperfect CSIR.
  arXiv  Detail & Related papers  (2022-05-02T12:23:55Z)
• Phase Configuration Learning in Wireless Networks with Multiple Reconfigurable Intelligent Surfaces [50.622375361505824]
  Reconfigurable Intelligent Surfaces (RISs) are highly scalable technology capable of offering dynamic control of electro-magnetic wave propagation. One of the major challenges with RIS-empowered wireless communications is the low-overhead dynamic configuration of multiple RISs. We devise low-complexity supervised learning approaches for the RISs' phase configurations.
  arXiv  Detail & Related papers  (2020-10-09T05:35:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.