Unicorn: A Universal and Collaborative Reinforcement Learning Approach Towards Generalizable Network-Wide Traffic Signal Control

• URL: http://arxiv.org/abs/2503.11488v1
• Date: Fri, 14 Mar 2025 15:13:42 GMT
• Title: Unicorn: A Universal and Collaborative Reinforcement Learning Approach Towards Generalizable Network-Wide Traffic Signal Control
• Authors: Yifeng Zhang, Yilin Liu, Ping Gong, Peizhuo Li, Mingfeng Fan, Guillaume Sartoretti,
• Abstract summary: Adaptive traffic signal control (ATSC) is crucial in reducing congestion, maximizing throughput, and improving mobility in rapidly growing urban areas.<n>Recent advancements in parameter-sharing multi-agent reinforcement learning (MARL) have greatly enhanced the scalable and adaptive optimization of complex, dynamic flows in large-scale homogeneous networks.<n>We present Unicorn, a universal and collaborative MARL framework designed for efficient and adaptable network-wide ATSC.
• Score: 13.106167353085878
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adaptive traffic signal control (ATSC) is crucial in reducing congestion, maximizing throughput, and improving mobility in rapidly growing urban areas. Recent advancements in parameter-sharing multi-agent reinforcement learning (MARL) have greatly enhanced the scalable and adaptive optimization of complex, dynamic flows in large-scale homogeneous networks. However, the inherent heterogeneity of real-world traffic networks, with their varied intersection topologies and interaction dynamics, poses substantial challenges to achieving scalable and effective ATSC across different traffic scenarios. To address these challenges, we present Unicorn, a universal and collaborative MARL framework designed for efficient and adaptable network-wide ATSC. Specifically, we first propose a unified approach to map the states and actions of intersections with varying topologies into a common structure based on traffic movements. Next, we design a Universal Traffic Representation (UTR) module with a decoder-only network for general feature extraction, enhancing the model's adaptability to diverse traffic scenarios. Additionally, we incorporate an Intersection Specifics Representation (ISR) module, designed to identify key latent vectors that represent the unique intersection's topology and traffic dynamics through variational inference techniques. To further refine these latent representations, we employ a contrastive learning approach in a self-supervised manner, which enables better differentiation of intersection-specific features. Moreover, we integrate the state-action dependencies of neighboring agents into policy optimization, which effectively captures dynamic agent interactions and facilitates efficient regional collaboration. Our results show that Unicorn outperforms other methods across various evaluation metrics, highlighting its potential in complex, dynamic traffic networks.

Related papers

• Federated Hierarchical Reinforcement Learning for Adaptive Traffic Signal Control [5.570882985800125]
  Multi-agent reinforcement learning (MARL) has shown promise for adaptive traffic signal control (ATSC) MARL faces constraints due to extensive data sharing and communication requirements. We propose Hierarchical Federated Reinforcement Learning (HFRL) for ATSC.
  arXiv  Detail & Related papers  (2025-04-07T23:02:59Z)
• CoLLMLight: Cooperative Large Language Model Agents for Network-Wide Traffic Signal Control [7.0964925117958515]
  Traffic Signal Control (TSC) plays a critical role in urban traffic management by optimizing traffic flow and mitigating congestion. Existing approaches fail to address the essential need for inter-agent coordination. We propose CoLLMLight, a cooperative LLM agent framework for TSC.
  arXiv  Detail & Related papers  (2025-03-14T15:40:39Z)
• Neighbor-Aware Reinforcement Learning for Mixed Traffic Optimization in Large-scale Networks [1.9413548770753521]
  This paper proposes a reinforcement learning framework for coordinating mixed traffic across interconnected intersections.<n>Our key contribution is a neighbor-aware reward mechanism that enables RVs to maintain balanced distribution across the network.<n>Results show that our method reduces average waiting times by 39.2% compared to the state-of-the-art single-intersection control policy.
  arXiv  Detail & Related papers  (2024-12-17T07:35:56Z)
• TransferLight: Zero-Shot Traffic Signal Control on any Road-Network [0.6274767633959003]
  TransferLight is a novel framework designed for robust generalization across road-networks.<n>Our hierarchical, heterogeneous, and directed graph neural network architecture effectively captures granular traffic dynamics.<n>We develop a single, weight-tied policy that scales zero-shot to any road network without re-training.
  arXiv  Detail & Related papers  (2024-12-12T20:52:12Z)
• Improving Traffic Flow Predictions with SGCN-LSTM: A Hybrid Model for Spatial and Temporal Dependencies [55.2480439325792]
  This paper introduces the Signal-Enhanced Graph Convolutional Network Long Short Term Memory (SGCN-LSTM) model for predicting traffic speeds across road networks. Experiments on the PEMS-BAY road network traffic dataset demonstrate the SGCN-LSTM model's effectiveness.
  arXiv  Detail & Related papers  (2024-11-01T00:37:00Z)
• CityLight: A Universal Model for Coordinated Traffic Signal Control in City-scale Heterogeneous Intersections [23.5766158697276]
  CityLight is a universal representation module that aligns the state representations of intersections and encodes narrowed relative traffic relation types to project the neighborhood intersections onto a uniform relative traffic impact space. Experiments with hundreds to tens of thousands of intersections validate the surprising effectiveness and generalizability of CityLight, with an overall performance gain of 11.68% and a 22.59% improvement in transfer scenarios in throughput.
  arXiv  Detail & Related papers  (2024-06-04T09:10:14Z)
• Adaptive Hierarchical SpatioTemporal Network for Traffic Forecasting [70.66710698485745]
  We propose an Adaptive Hierarchical SpatioTemporal Network (AHSTN) to promote traffic forecasting. AHSTN exploits the spatial hierarchy and modeling multi-scale spatial correlations. Experiments on two real-world datasets show that AHSTN achieves better performance over several strong baselines.
  arXiv  Detail & Related papers  (2023-06-15T14:50:27Z)
• Cross-receptive Focused Inference Network for Lightweight Image Super-Resolution [64.25751738088015]
  Transformer-based methods have shown impressive performance in single image super-resolution (SISR) tasks. Transformers that need to incorporate contextual information to extract features dynamically are neglected. We propose a lightweight Cross-receptive Focused Inference Network (CFIN) that consists of a cascade of CT Blocks mixed with CNN and Transformer.
  arXiv  Detail & Related papers  (2022-07-06T16:32:29Z)
• Feudal Multi-Agent Reinforcement Learning with Adaptive Network Partition for Traffic Signal Control [44.09601435685123]
  Multi-agent reinforcement learning (MARL) has been applied and shown great potential in traffic signal control. Previous work partitions the traffic network into several regions and learns policies for agents in a feudal structure. We propose a novel feudal MARL approach with adaptive network partition.
  arXiv  Detail & Related papers  (2022-05-27T09:02:10Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline [85.9210953301628]
  Control of traffic signals is fundamental and critical to alleviate traffic congestion in urban areas. Because of the high complexity of modelling the problem, experimental settings of current works are often inconsistent. We propose a novel and strong baseline model based on deep reinforcement learning with the encoder-decoder structure.
  arXiv  Detail & Related papers  (2021-01-24T03:55:39Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)
• Multi-Agent Routing Value Iteration Network [88.38796921838203]
  We propose a graph neural network based model that is able to perform multi-agent routing based on learned value in a sparsely connected graph. We show that our model trained with only two agents on graphs with a maximum of 25 nodes can easily generalize to situations with more agents and/or nodes.
  arXiv  Detail & Related papers  (2020-07-09T22:16:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.