Learning to Reflect: Hierarchical Multi-Agent Reinforcement Learning for CSI-Free mmWave Beam-Focusing

• URL: http://arxiv.org/abs/2603.07370v1
• Date: Sat, 07 Mar 2026 22:46:21 GMT
• Title: Learning to Reflect: Hierarchical Multi-Agent Reinforcement Learning for CSI-Free mmWave Beam-Focusing
• Abstract summary: We introduce a "CSI-free" paradigm that substitutes pilot-based channel estimation with readily available user localization data.<n>The proposed architecture decomposes the control into two levels: a high-level controller for user-to-reflector allocation and decentralized low-level controllers for low-level focal point optimization.
• Score: 8.046649921554472
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reconfigurable Intelligent Surfaces promise to transform wireless environments, yet practical deployment is hindered by the prohibitive overhead of Channel State Information (CSI) estimation and the dimensionality explosion inherent in centralized optimization. This paper proposes a Hierarchical Multi-Agent Reinforcement Learning (HMARL) framework for the control of mechanically reconfigurable reflective surfaces in millimeter-wave (mmWave) systems. We introduce a "CSI-free" paradigm that substitutes pilot-based channel estimation with readily available user localization data. To manage the massive combinatorial action space, the proposed architecture utilizes Multi-Agent Proximal Policy Optimization (MAPPO) under a Centralized Training with Decentralized Execution (CTDE) paradigm. The proposed architecture decomposes the control problem into two abstraction levels: a high-level controller for user-to-reflector allocation and decentralized low-level controllers for low-level focal point optimization. Comprehensive ray-tracing evaluations demonstrate that the framework achieves 2.81-7.94 dB RSSI improvements over centralized baselines, with the performance advantage widening as system complexity increases. Scalability analysis reveals that the system maintains sustained efficiency, exhibiting minimal per-user performance degradation and stable total power utilization even when user density doubles. Furthermore, robustness validation confirms the framework's viability across varying reflector aperture sizes (45-99 tiles) and demonstrates graceful performance degradation under localization errors up to 0.5 m. By eliminating CSI overhead while maintaining high-fidelity beam-focusing, this work establishes HMARL as a practical solution for intelligent mmWave environments.

Related papers

• Learning to Focus: CSI-Free Hierarchical MARL for Reconfigurable Reflectors [8.046649921554472]
  Reconfigurable Intelligent Surfaces (RIS) has a potential to engineer smart radio environments for next-generation millimeter-wave (mmWave) networks.<n>We introduce a CSI-free" paradigm powered by a Hierarchical Multi-Agent Reinforcement Learning (HMARL) architecture to control mechanically reconfigurable surfaces.
  arXiv  Detail & Related papers  (2026-04-06T20:50:03Z)
• Bypassing the CSI Bottleneck: MARL-Driven Spatial Control for Reflector Arrays [8.046649921554472]
  This paper presents a fully autonomous Multi-Agent Reinforcement Learning (MARL) framework to control mechanically adjustable metallic reflector arrays.<n>High-fidelity ray-tracing simulations in dynamic non-line-of-sight (NLOS) environments demonstrate that this multi-agent approach rapidly adapts to user mobility.<n>These results validate the efficacy of MARL-driven spatial abstractions as a scalable, highly practical pathway toward AI-empowered wireless networks.
  arXiv  Detail & Related papers  (2026-04-06T20:47:50Z)
• Efficient Image Super-Resolution with Multi-Scale Spatial Adaptive Attention Networks [3.4782736103257323]
  This paper introduces a lightweight image super-resolution (SR) network, termed the Multi-scale Spatial Adaptive Attention Network (MSAAN)<n>The core of our approach is a novel Multi-scale Spatial Adaptive Attention Module (MSAA), designed to jointly model fine-grained local details and long-range contextual dependencies.
  arXiv  Detail & Related papers  (2026-02-22T07:47:39Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• HypeMARL: Multi-Agent Reinforcement Learning For High-Dimensional, Parametric, and Distributed Systems [3.072554747025686]
  HypeMARL is a decentralized reinforcement learning algorithm tailored to the control of high-dimensional, parametric, and distributed systems.<n>We show that HypeMARL can effectively control systems through a collective behavior of the agents, outperforming state-of-the-art decentralized MARL.
  arXiv  Detail & Related papers  (2025-09-20T14:42:09Z)
• Joint Channel Estimation and Computation Offloading in Fluid Antenna-assisted MEC Networks [81.36647816787713]
  We propose an FA-assisted offloading framework to minimize the delay of channel estimation.<n>We show that the proposed system significantly reduces the accuracy under efficient communication.
  arXiv  Detail & Related papers  (2025-09-16T08:48:44Z)
• CSIYOLO: An Intelligent CSI-based Scatter Sensing Framework for Integrated Sensing and Communication Systems [18.244924566345027]
  ISAC is regarded as a promising technology for next-generation communication systems, enabling simultaneous data transmission and target sensing.<n> scatter sensing plays a crucial role in exploiting the full potential of ISAC and supporting applications such as autonomous driving and low-altitude economy.<n>We propose CSIYOLO, a framework that performs scatter localization using estimated CSI from a single base station-user equipment pair.
  arXiv  Detail & Related papers  (2025-09-15T06:46:39Z)
• Green Learning for STAR-RIS mmWave Systems with Implicit CSI [53.03358325565645]
  Green learning (GL)-based precoding framework is proposed for simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS)-aided millimeter-wave (mmWave) broadcasting systems.<n>Motivated by the emphasis on environmental sustainability in future 6G networks, this work adopts a transmission framework for scenarios where multiple users share identical information, improving spectral efficiency and reducing redundant transmissions and power consumption.
  arXiv  Detail & Related papers  (2025-09-08T15:56:06Z)
• Backscatter Device-aided Integrated Sensing and Communication: A Pareto Optimization Framework [59.30060797118097]
  Integrated sensing and communication (ISAC) systems potentially encounter significant performance degradation in densely obstructed urban non-line-of-sight scenarios.<n>This paper proposes a backscatter approximation (BD)-assisted ISAC system, which leverages passive BDs naturally distributed in environments of enhancement.
  arXiv  Detail & Related papers  (2025-07-12T17:11:06Z)
• Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach [51.63921041249406]
  Non-orthogonal multiple access (NOMA) enables multiple users to share the same frequency band, and simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) deploying STAR-RIS indoors presents challenges in interference mitigation, power consumption, and real-time configuration. A novel network architecture utilizing multiple access points (APs), STAR-RISs, and NOMA is proposed for indoor communication.
  arXiv  Detail & Related papers  (2024-06-19T07:17:04Z)
• Digital Twin-Aided Learning for Managing Reconfigurable Intelligent Surface-Assisted, Uplink, User-Centric Cell-Free Systems [34.81029978058933]
  This paper puts forth a new, reconfigurable intelligent surface (RIS)-assisted, uplink, user-centric cell-free (UCCF) system managed with the assistance of a digital twin (DT) Specifically, we propose a novel learning framework that maximizes the sum-rate by jointly optimizing the access point and user association (AUA), power control, and RIS beamforming.
  arXiv  Detail & Related papers  (2023-02-10T06:14:47Z)
• Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning [63.83425382922157]
  Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting.
  arXiv  Detail & Related papers  (2022-03-26T20:37:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.