A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

• URL: http://arxiv.org/abs/2107.06115v1
• Date: Tue, 13 Jul 2021 14:11:04 GMT
• Title: A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization
• Authors: Zhenning Li, Chengzhong Xu, Guohui Zhang
• Abstract summary: Inefficient traffic signal control methods may cause numerous problems, such as traffic congestion and waste of energy. This paper first proposes a multi-agent deep deterministic policy gradient (MADDPG) method by extending the actor-critic policy gradient algorithms.
• Score: 14.455497228170646
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inefficient traffic signal control methods may cause numerous problems, such as traffic congestion and waste of energy. Reinforcement learning (RL) is a trending data-driven approach for adaptive traffic signal control in complex urban traffic networks. Although the development of deep neural networks (DNN) further enhances its learning capability, there are still some challenges in applying deep RLs to transportation networks with multiple signalized intersections, including non-stationarity environment, exploration-exploitation dilemma, multi-agent training schemes, continuous action spaces, etc. In order to address these issues, this paper first proposes a multi-agent deep deterministic policy gradient (MADDPG) method by extending the actor-critic policy gradient algorithms. MADDPG has a centralized learning and decentralized execution paradigm in which critics use additional information to streamline the training process, while actors act on their own local observations. The model is evaluated via simulation on the Simulation of Urban MObility (SUMO) platform. Model comparison results show the efficiency of the proposed algorithm in controlling traffic lights.

Related papers

• Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion [2.733700237741334]
  This paper explores the use of Reinforcement Learning to enhance traffic signal operations at intersections. We introduce two RL-based algorithms: a turn-based agent, which dynamically prioritizes traffic signals based on real-time queue lengths, and a time-based agent, which adjusts signal phase durations according to traffic conditions. Simulation results demonstrate that both RL algorithms significantly outperform conventional traffic signal control systems.
  arXiv  Detail & Related papers  (2024-08-28T12:35:56Z)
• A Holistic Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation [53.39174966020085]
  Traffic signal control (TSC) is crucial for reducing traffic congestion that leads to smoother traffic flow, reduced idling time, and mitigated CO2 emissions. In this study, we explore the computer vision approach for TSC that modulates on-road traffic flows through visual observation. We introduce a holistic traffic simulation framework called TrafficDojo towards vision-based TSC and its benchmarking.
  arXiv  Detail & Related papers  (2024-03-11T16:42:29Z)
• Learning Decentralized Traffic Signal Controllers with Multi-Agent Graph Reinforcement Learning [42.175067773481416]
  We design a new decentralized control architecture with improved environmental observability to capture the spatial-temporal correlation. Specifically, we first develop a topology-aware information aggregation strategy to extract correlation-related information from unstructured data gathered in the road network. A diffusion convolution module is developed, forming a new MARL algorithm, which endows agents with the capabilities of graph learning.
  arXiv  Detail & Related papers  (2023-11-07T06:43:15Z)
• A Novel Multi-Agent Deep RL Approach for Traffic Signal Control [13.927155702352131]
  We propose a Friend-Deep Q-network (Friend-DQN) approach for multiple traffic signal control in urban networks. In particular, the cooperation between multiple agents can reduce the state-action space and thus speed up the convergence.
  arXiv  Detail & Related papers  (2023-06-05T08:20:37Z)
• Improving the generalizability and robustness of large-scale traffic signal control [3.8028221877086814]
  We study the robustness of deep reinforcement-learning (RL) approaches to control traffic signals. We show that recent methods remain brittle in the face of missing data. We propose using a combination of distributional and vanilla reinforcement learning through a policy ensemble.
  arXiv  Detail & Related papers  (2023-06-02T21:30:44Z)
• AI-aided Traffic Control Scheme for M2M Communications in the Internet of Vehicles [61.21359293642559]
  The dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies. We consider a hybrid traffic control scheme and use proximal policy optimization (PPO) method to tackle it.
  arXiv  Detail & Related papers  (2022-03-05T10:54:05Z)
• Road Network Guided Fine-Grained Urban Traffic Flow Inference [108.64631590347352]
  Accurate inference of fine-grained traffic flow from coarse-grained one is an emerging yet crucial problem. We propose a novel Road-Aware Traffic Flow Magnifier (RATFM) that exploits the prior knowledge of road networks. Our method can generate high-quality fine-grained traffic flow maps.
  arXiv  Detail & Related papers  (2021-09-29T07:51:49Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline [85.9210953301628]
  Control of traffic signals is fundamental and critical to alleviate traffic congestion in urban areas. Because of the high complexity of modelling the problem, experimental settings of current works are often inconsistent. We propose a novel and strong baseline model based on deep reinforcement learning with the encoder-decoder structure.
  arXiv  Detail & Related papers  (2021-01-24T03:55:39Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)
• IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control [4.273991039651846]
  Scaling adaptive traffic-signal control involves dealing with state and action spaces. We introduce Inductive Graph Reinforcement Learning (IG-RL) based on graph-convolutional networks. Our model can generalize to new road networks, traffic distributions, and traffic regimes.
  arXiv  Detail & Related papers  (2020-03-06T17:17:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.