Reinforcement Learning Based Traffic Signal Design to Minimize Queue Lengths

• URL: http://arxiv.org/abs/2509.21745v1
• Date: Fri, 26 Sep 2025 01:23:30 GMT
• Title: Reinforcement Learning Based Traffic Signal Design to Minimize Queue Lengths
• Authors: Anirud Nandakumar, Chayan Banerjee, Lelitha Devi Vanajakshi,
• Abstract summary: We propose a novel adaptive TSC framework that leverages Reinforcement Learning (RL) to minimize total queue lengths across all signal phases.<n>The proposed algorithm has been implemented using the Simulation of Urban Mobility (SUMO) traffic simulator.<n>The best performing configuration achieves an approximately 29% reduction in average queue lengths compared to the traditional Webster method.
• Score: 15.439906983758808
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Efficient traffic signal control (TSC) is crucial for reducing congestion, travel delays, pollution, and for ensuring road safety. Traditional approaches, such as fixed signal control and actuated control, often struggle to handle dynamic traffic patterns. In this study, we propose a novel adaptive TSC framework that leverages Reinforcement Learning (RL), using the Proximal Policy Optimization (PPO) algorithm, to minimize total queue lengths across all signal phases. The challenge of efficiently representing highly stochastic traffic conditions for an RL controller is addressed through multiple state representations, including an expanded state space, an autoencoder representation, and a K-Planes-inspired representation. The proposed algorithm has been implemented using the Simulation of Urban Mobility (SUMO) traffic simulator and demonstrates superior performance over both traditional methods and other conventional RL-based approaches in reducing queue lengths. The best performing configuration achieves an approximately 29% reduction in average queue lengths compared to the traditional Webster method. Furthermore, comparative evaluation of alternative reward formulations demonstrates the effectiveness of the proposed queue-based approach, showcasing the potential for scalable and adaptive urban traffic management.

Related papers

• Quantifying Distribution Shift in Traffic Signal Control with Histogram-Based GEH Distance [0.6961253535504978]
  This paper introduces a principled approach to quantify distribution shift by representing traffic scenarios as demand histograms.<n>We validate the approach on 20 simulated scenarios using both a NEMA actuated controller and a reinforcement learning controller.<n>Overall, this method can predict performance degradation under distribution shift better than previously published techniques.
  arXiv  Detail & Related papers  (2025-11-16T11:05:02Z)
• Robust Single-Agent Reinforcement Learning for Regional Traffic Signal Control Under Demand Fluctuations [5.784337914162491]
  Traffic congestion, primarily driven by intersection queuing, significantly impacts urban living standards, safety, environmental quality, and economic efficiency.<n>This study introduces a novel single-agent reinforcement learning framework for regional adaptive TSC.<n>The framework exhibits robust anti-fluctuation capability and significantly reduces queue lengths.
  arXiv  Detail & Related papers  (2025-11-01T13:18:50Z)
• Improving Traffic Flow Predictions with SGCN-LSTM: A Hybrid Model for Spatial and Temporal Dependencies [55.2480439325792]
  This paper introduces the Signal-Enhanced Graph Convolutional Network Long Short Term Memory (SGCN-LSTM) model for predicting traffic speeds across road networks. Experiments on the PEMS-BAY road network traffic dataset demonstrate the SGCN-LSTM model's effectiveness.
  arXiv  Detail & Related papers  (2024-11-01T00:37:00Z)
• Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion [2.733700237741334]
  This paper explores the use of Reinforcement Learning to enhance traffic signal operations at intersections. We introduce two RL-based algorithms: a turn-based agent, which dynamically prioritizes traffic signals based on real-time queue lengths, and a time-based agent, which adjusts signal phase durations according to traffic conditions. Simulation results demonstrate that both RL algorithms significantly outperform conventional traffic signal control systems.
  arXiv  Detail & Related papers  (2024-08-28T12:35:56Z)
• A Holistic Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation [53.39174966020085]
  Traffic signal control (TSC) is crucial for reducing traffic congestion that leads to smoother traffic flow, reduced idling time, and mitigated CO2 emissions. In this study, we explore the computer vision approach for TSC that modulates on-road traffic flows through visual observation. We introduce a holistic traffic simulation framework called TrafficDojo towards vision-based TSC and its benchmarking.
  arXiv  Detail & Related papers  (2024-03-11T16:42:29Z)
• Reinforcement Learning with Model Predictive Control for Highway Ramp Metering [14.389086937116582]
  This work explores the synergy between model-based and learning-based strategies to enhance traffic flow management.<n>The control problem is formulated as an RL task by crafting a suitable stage cost function.<n>An MPC-based RL approach, which leverages the MPC optimal problem as a function approximation for the RL algorithm, is proposed to learn to efficiently control an on-ramp.
  arXiv  Detail & Related papers  (2023-11-15T09:50:54Z)
• DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback [109.84667902348498]
  Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network. Most prior TSC methods leverage deep reinforcement learning to search for a control policy. We propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness.
  arXiv  Detail & Related papers  (2023-06-13T05:58:57Z)
• AI-aided Traffic Control Scheme for M2M Communications in the Internet of Vehicles [61.21359293642559]
  The dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies. We consider a hybrid traffic control scheme and use proximal policy optimization (PPO) method to tackle it.
  arXiv  Detail & Related papers  (2022-03-05T10:54:05Z)
• Efficient Pressure: Improving efficiency for signalized intersections [24.917612761503996]
  Reinforcement learning (RL) has attracted more attention to help solve the traffic signal control (TSC) problem. Existing RL-based methods are rarely deployed considering that they are neither cost-effective in terms of computing resources nor more robust than traditional approaches. We demonstrate how to construct an adaptive controller for TSC with less training and reduced complexity based on RL-based approach.
  arXiv  Detail & Related papers  (2021-12-04T13:49:58Z)
• Road Network Guided Fine-Grained Urban Traffic Flow Inference [108.64631590347352]
  Accurate inference of fine-grained traffic flow from coarse-grained one is an emerging yet crucial problem. We propose a novel Road-Aware Traffic Flow Magnifier (RATFM) that exploits the prior knowledge of road networks. Our method can generate high-quality fine-grained traffic flow maps.
  arXiv  Detail & Related papers  (2021-09-29T07:51:49Z)
• A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization [14.455497228170646]
  Inefficient traffic signal control methods may cause numerous problems, such as traffic congestion and waste of energy. This paper first proposes a multi-agent deep deterministic policy gradient (MADDPG) method by extending the actor-critic policy gradient algorithms.
  arXiv  Detail & Related papers  (2021-07-13T14:11:04Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.