Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning

• URL: http://arxiv.org/abs/2311.03756v1
• Date: Tue, 7 Nov 2023 06:43:15 GMT
• Title: Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning
• Authors: Yao Zhang, Zhiwen Yu, Jun Zhang, Liang Wang, Tom H. Luan, Bin Guo, Chau Yuen
• Abstract summary: We design a new decentralized control architecture with improved environmental observability to capture the spatial-temporal correlation. Specifically, we first develop a topology-aware information aggregation strategy to extract correlation-related information from unstructured data gathered in the road network. A diffusion convolution module is developed, forming a new MARL algorithm, which endows agents with the capabilities of graph learning.
• Score: 42.175067773481416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper considers optimal traffic signal control in smart cities, which has been taken as a complex networked system control problem. Given the interacting dynamics among traffic lights and road networks, attaining controller adaptivity and scalability stands out as a primary challenge. Capturing the spatial-temporal correlation among traffic lights under the framework of Multi-Agent Reinforcement Learning (MARL) is a promising solution. Nevertheless, existing MARL algorithms ignore effective information aggregation which is fundamental for improving the learning capacity of decentralized agents. In this paper, we design a new decentralized control architecture with improved environmental observability to capture the spatial-temporal correlation. Specifically, we first develop a topology-aware information aggregation strategy to extract correlation-related information from unstructured data gathered in the road network. Particularly, we transfer the road network topology into a graph shift operator by forming a diffusion process on the topology, which subsequently facilitates the construction of graph signals. A diffusion convolution module is developed, forming a new MARL algorithm, which endows agents with the capabilities of graph learning. Extensive experiments based on both synthetic and real-world datasets verify that our proposal outperforms existing decentralized algorithms.

Related papers

• Improving Traffic Flow Predictions with SGCN-LSTM: A Hybrid Model for Spatial and Temporal Dependencies [55.2480439325792]
  This paper introduces the Signal-Enhanced Graph Convolutional Network Long Short Term Memory (SGCN-LSTM) model for predicting traffic speeds across road networks. Experiments on the PEMS-BAY road network traffic dataset demonstrate the SGCN-LSTM model's effectiveness.
  arXiv  Detail & Related papers  (2024-11-01T00:37:00Z)
• Towards Multi-agent Reinforcement Learning based Traffic Signal Control through Spatio-temporal Hypergraphs [19.107744041461316]
  Traffic signal control systems (TSCSs) are integral to intelligent traffic management, fostering efficient vehicle flow. Traditional approaches often simplify road networks into standard graphs. We propose a novel TSCS framework to realize intelligent traffic control.
  arXiv  Detail & Related papers  (2024-04-17T02:46:18Z)
• Learning State-Augmented Policies for Information Routing in Communication Networks [92.59624401684083]
  We develop a novel State Augmentation (SA) strategy to maximize the aggregate information at source nodes using graph neural network (GNN) architectures. We leverage an unsupervised learning procedure to convert the output of the GNN architecture to optimal information routing strategies. In the experiments, we perform the evaluation on real-time network topologies to validate our algorithms.
  arXiv  Detail & Related papers  (2023-09-30T04:34:25Z)
• Adaptive Hierarchical SpatioTemporal Network for Traffic Forecasting [70.66710698485745]
  We propose an Adaptive Hierarchical SpatioTemporal Network (AHSTN) to promote traffic forecasting. AHSTN exploits the spatial hierarchy and modeling multi-scale spatial correlations. Experiments on two real-world datasets show that AHSTN achieves better performance over several strong baselines.
  arXiv  Detail & Related papers  (2023-06-15T14:50:27Z)
• Improving the generalizability and robustness of large-scale traffic signal control [3.8028221877086814]
  We study the robustness of deep reinforcement-learning (RL) approaches to control traffic signals. We show that recent methods remain brittle in the face of missing data. We propose using a combination of distributional and vanilla reinforcement learning through a policy ensemble.
  arXiv  Detail & Related papers  (2023-06-02T21:30:44Z)
• DMGCRN: Dynamic Multi-Graph Convolution Recurrent Network for Traffic Forecasting [7.232141271583618]
  We propose a novel dynamic multi-graph convolution recurrent network (DMG) to tackle above issues. We use the distance-based graph to capture spatial information from nodes are close in distance. We also construct a novel latent graph which encoded the structure correlations among roads to capture spatial information from nodes are similar in structure.
  arXiv  Detail & Related papers  (2021-12-04T06:51:55Z)
• A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization [14.455497228170646]
  Inefficient traffic signal control methods may cause numerous problems, such as traffic congestion and waste of energy. This paper first proposes a multi-agent deep deterministic policy gradient (MADDPG) method by extending the actor-critic policy gradient algorithms.
  arXiv  Detail & Related papers  (2021-07-13T14:11:04Z)
• Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline [85.9210953301628]
  Control of traffic signals is fundamental and critical to alleviate traffic congestion in urban areas. Because of the high complexity of modelling the problem, experimental settings of current works are often inconsistent. We propose a novel and strong baseline model based on deep reinforcement learning with the encoder-decoder structure.
  arXiv  Detail & Related papers  (2021-01-24T03:55:39Z)
• A Compressive Sensing Approach for Federated Learning over Massive MIMO Communication Systems [82.2513703281725]
  Federated learning is a privacy-preserving approach to train a global model at a central server by collaborating with wireless devices. We present a compressive sensing approach for federated learning over massive multiple-input multiple-output communication systems.
  arXiv  Detail & Related papers  (2020-03-18T05:56:27Z)
• IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control [4.273991039651846]
  Scaling adaptive traffic-signal control involves dealing with state and action spaces. We introduce Inductive Graph Reinforcement Learning (IG-RL) based on graph-convolutional networks. Our model can generalize to new road networks, traffic distributions, and traffic regimes.
  arXiv  Detail & Related papers  (2020-03-06T17:17:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.