HiLight: A Hierarchical Reinforcement Learning Framework with Global Adversarial Guidance for Large-Scale Traffic Signal Control

• URL: http://arxiv.org/abs/2506.14391v1
• Date: Tue, 17 Jun 2025 10:39:42 GMT
• Title: HiLight: A Hierarchical Reinforcement Learning Framework with Global Adversarial Guidance for Large-Scale Traffic Signal Control
• Authors: Yaqiao Zhu, Hongkai Wen, Geyong Min, Man Luo,
• Abstract summary: HiLight is a hierarchical reinforcement learning framework with global adversarial guidance for large-scale traffic signal control.<n>We evaluate HiLight across both synthetic and real-world benchmarks, and construct a large-scale Manhattan network with diverse traffic conditions.
• Score: 22.185954207474907
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Efficient traffic signal control (TSC) is essential for mitigating urban congestion, yet existing reinforcement learning (RL) methods face challenges in scaling to large networks while maintaining global coordination. Centralized RL suffers from scalability issues, while decentralized approaches often lack unified objectives, resulting in limited network-level efficiency. In this paper, we propose HiLight, a hierarchical reinforcement learning framework with global adversarial guidance for large-scale TSC. HiLight consists of a high-level Meta-Policy, which partitions the traffic network into subregions and generates sub-goals using a Transformer-LSTM architecture, and a low-level Sub-Policy, which controls individual intersections with global awareness. To improve the alignment between global planning and local execution, we introduce an adversarial training mechanism, where the Meta-Policy generates challenging yet informative sub-goals, and the Sub-Policy learns to surpass these targets, leading to more effective coordination. We evaluate HiLight across both synthetic and real-world benchmarks, and additionally construct a large-scale Manhattan network with diverse traffic conditions, including peak transitions, adverse weather, and holiday surges. Experimental results show that HiLight exhibits significant advantages in large-scale scenarios and remains competitive across standard benchmarks of varying sizes.

Related papers

• World Model-Based Learning for Long-Term Age of Information Minimization in Vehicular Networks [53.98633183204453]
  In this paper, a novel world model-based learning framework is proposed to minimize packet-completeness-aware age of information (CAoI) in a vehicular network.<n>A world model framework is proposed to jointly learn a dynamic model of the mmWave V2X environment and use it to imagine trajectories for learning how to perform link scheduling.<n>In particular, the long-term policy is learned in differentiable imagined trajectories instead of environment interactions.
  arXiv  Detail & Related papers  (2025-05-03T06:23:18Z)
• Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control [5.570882985800125]
  Multi-agent reinforcement learning (MARL) has shown promise for adaptive traffic signal control (ATSC)<n>MARL faces constraints due to extensive data sharing and communication requirements.<n>We propose Hierarchical Federated Reinforcement Learning (HFRL) for ATSC.
  arXiv  Detail & Related papers  (2025-04-07T23:02:59Z)
• CoLLMLight: Cooperative Large Language Model Agents for Network-Wide Traffic Signal Control [7.0964925117958515]
  Traffic Signal Control (TSC) plays a critical role in urban traffic management by optimizing traffic flow and mitigating congestion.<n>Existing approaches fail to address the essential need for inter-agent coordination.<n>We propose CoLLMLight, a cooperative LLM agent framework for TSC.
  arXiv  Detail & Related papers  (2025-03-14T15:40:39Z)
• Toward Dependency Dynamics in Multi-Agent Reinforcement Learning for Traffic Signal Control [8.312659530314937]
  Reinforcement learning (RL) emerges as a promising data-driven approach for adaptive traffic signal control.<n>In this paper, we propose a novel Dynamic Reinforcement Update Strategy for Deep Q-Network (DQN-DPUS)<n>We show that the proposed strategy can speed up the convergence rate without sacrificing optimal exploration.
  arXiv  Detail & Related papers  (2025-02-23T15:29:12Z)
• Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach [51.63921041249406]
  Non-orthogonal multiple access (NOMA) enables multiple users to share the same frequency band, and simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) deploying STAR-RIS indoors presents challenges in interference mitigation, power consumption, and real-time configuration. A novel network architecture utilizing multiple access points (APs), STAR-RISs, and NOMA is proposed for indoor communication.
  arXiv  Detail & Related papers  (2024-06-19T07:17:04Z)
• CityLight: A Universal Model for Coordinated Traffic Signal Control in City-scale Heterogeneous Intersections [23.5766158697276]
  CityLight is a universal representation module that aligns the state representations of intersections and encodes narrowed relative traffic relation types to project the neighborhood intersections onto a uniform relative traffic impact space. Experiments with hundreds to tens of thousands of intersections validate the surprising effectiveness and generalizability of CityLight, with an overall performance gain of 11.68% and a 22.59% improvement in transfer scenarios in throughput.
  arXiv  Detail & Related papers  (2024-06-04T09:10:14Z)
• LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environments [3.7788636451616697]
  This work introduces an innovative approach that integrates Large Language Models into traffic signal control systems. A hybrid framework that augments LLMs with a suite of perception and decision-making tools is proposed. The findings from our simulations attest to the system's adeptness in adjusting to a multiplicity of traffic environments.
  arXiv  Detail & Related papers  (2024-03-13T08:41:55Z)
• SINR-Aware Deep Reinforcement Learning for Distributed Dynamic Channel Allocation in Cognitive Interference Networks [10.514231683620517]
  This paper focuses on real-world systems experiencing inter-carrier interference (ICI) and channel reuse by multiple large-scale networks. We propose a novel multi-agent reinforcement learning framework for distributed DCA, named Channel Allocation RL To Overlapped Networks (CARLTON) Our results demonstrate exceptional performance and robust generalization, showcasing superior efficiency compared to alternative state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-17T20:03:02Z)
• Learning to Sail Dynamic Networks: The MARLIN Reinforcement Learning Framework for Congestion Control in Tactical Environments [53.08686495706487]
  This paper proposes an RL framework that leverages an accurate and parallelizable emulation environment to reenact the conditions of a tactical network. We evaluate our RL learning framework by training a MARLIN agent in conditions replicating a bottleneck link transition between a Satellite Communication (SATCOM) and an UHF Wide Band (UHF) radio link.
  arXiv  Detail & Related papers  (2023-06-27T16:15:15Z)
• Adaptive Hierarchical SpatioTemporal Network for Traffic Forecasting [70.66710698485745]
  We propose an Adaptive Hierarchical SpatioTemporal Network (AHSTN) to promote traffic forecasting. AHSTN exploits the spatial hierarchy and modeling multi-scale spatial correlations. Experiments on two real-world datasets show that AHSTN achieves better performance over several strong baselines.
  arXiv  Detail & Related papers  (2023-06-15T14:50:27Z)
• MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks [63.24965775030673]
  We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
  arXiv  Detail & Related papers  (2023-02-02T18:27:20Z)
• Cross-receptive Focused Inference Network for Lightweight Image Super-Resolution [64.25751738088015]
  Transformer-based methods have shown impressive performance in single image super-resolution (SISR) tasks. Transformers that need to incorporate contextual information to extract features dynamically are neglected. We propose a lightweight Cross-receptive Focused Inference Network (CFIN) that consists of a cascade of CT Blocks mixed with CNN and Transformer.
  arXiv  Detail & Related papers  (2022-07-06T16:32:29Z)
• Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learning [52.7873574425376]
  Cooperative multi-agent reinforcement learning (MARL) faces significant scalability issues due to state and action spaces that are exponentially large in the number of agents. We propose a novel, value-based multi-agent algorithm called LOMAQ, which incorporates local rewards in the Training Decentralized Execution paradigm.
  arXiv  Detail & Related papers  (2021-09-22T10:08:15Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.