Enhancing Traffic Signal Control through Model-based Reinforcement Learning and Policy Reuse

• URL: http://arxiv.org/abs/2503.08728v1
• Date: Tue, 11 Mar 2025 01:21:13 GMT
• Title: Enhancing Traffic Signal Control through Model-based Reinforcement Learning and Policy Reuse
• Authors: Yihong Li, Chengwei Zhang, Furui Zhan, Wanting Liu, Kailing Zhou, Longji Zheng,
• Abstract summary: Multi-agent reinforcement learning (MARL) has shown significant potential in traffic signal control (TSC)<n>Current MARL-based methods often suffer from insufficient generalization due to the fixed traffic patterns and road network conditions used during training.<n>This limitation results in poor adaptability to new traffic scenarios, leading to high retraining costs and complex deployment.<n>We propose two algorithms: PLight and PRLight. PLight employs a model-based reinforcement learning approach, pretraining control policies and environment models using predefined source-domain traffic scenarios. PRLight further enhances adaptability by adaptively selecting pre-trained PLight agents based on the similarity between
• Score: 0.9995933996287355
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-agent reinforcement learning (MARL) has shown significant potential in traffic signal control (TSC). However, current MARL-based methods often suffer from insufficient generalization due to the fixed traffic patterns and road network conditions used during training. This limitation results in poor adaptability to new traffic scenarios, leading to high retraining costs and complex deployment. To address this challenge, we propose two algorithms: PLight and PRLight. PLight employs a model-based reinforcement learning approach, pretraining control policies and environment models using predefined source-domain traffic scenarios. The environment model predicts the state transitions, which facilitates the comparison of environmental features. PRLight further enhances adaptability by adaptively selecting pre-trained PLight agents based on the similarity between the source and target domains to accelerate the learning process in the target domain. We evaluated the algorithms through two transfer settings: (1) adaptability to different traffic scenarios within the same road network, and (2) generalization across different road networks. The results show that PRLight significantly reduces the adaptation time compared to learning from scratch in new TSC scenarios, achieving optimal performance using similarities between available and target scenarios.

Related papers

• Strada-LLM: Graph LLM for traffic prediction [62.2015839597764]
  A considerable challenge in traffic prediction lies in handling the diverse data distributions caused by vastly different traffic conditions.<n>We propose a graph-aware LLM for traffic prediction that considers proximal traffic information.<n>We adopt a lightweight approach for efficient domain adaptation when facing new data distributions in few-shot fashion.
  arXiv  Detail & Related papers  (2024-10-28T09:19:29Z)
• Reinforcement Learning for Adaptive Traffic Signal Control: Turn-Based and Time-Based Approaches to Reduce Congestion [2.733700237741334]
  This paper explores the use of Reinforcement Learning to enhance traffic signal operations at intersections. We introduce two RL-based algorithms: a turn-based agent, which dynamically prioritizes traffic signals based on real-time queue lengths, and a time-based agent, which adjusts signal phase durations according to traffic conditions. Simulation results demonstrate that both RL algorithms significantly outperform conventional traffic signal control systems.
  arXiv  Detail & Related papers  (2024-08-28T12:35:56Z)
• A Holistic Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation [53.39174966020085]
  Traffic signal control (TSC) is crucial for reducing traffic congestion that leads to smoother traffic flow, reduced idling time, and mitigated CO2 emissions. In this study, we explore the computer vision approach for TSC that modulates on-road traffic flows through visual observation. We introduce a holistic traffic simulation framework called TrafficDojo towards vision-based TSC and its benchmarking.
  arXiv  Detail & Related papers  (2024-03-11T16:42:29Z)
• SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control [7.387226437589183]
  SocialLight is a new multi-agent reinforcement learning method for traffic signal control. It learns cooperative traffic control policies by estimating the individual marginal contribution of agents on their local neighborhood. We benchmark our trained network against state-of-the-art traffic signal control methods on standard benchmarks in two traffic simulators.
  arXiv  Detail & Related papers  (2023-04-20T12:41:25Z)
• AI-aided Traffic Control Scheme for M2M Communications in the Internet of Vehicles [61.21359293642559]
  The dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies. We consider a hybrid traffic control scheme and use proximal policy optimization (PPO) method to tackle it.
  arXiv  Detail & Related papers  (2022-03-05T10:54:05Z)
• Intelligent Traffic Light via Policy-based Deep Reinforcement Learning [2.0796717061432]
  In this study, we employ reinforcement learning to train the control agent of a traffic light on a simulator of urban mobility. As a difference from existing works, a policy-based deep reinforcement learning method, Proximal Policy Optimization (PPO), is utilized.
  arXiv  Detail & Related papers  (2021-12-27T18:11:32Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• AdaPool: A Diurnal-Adaptive Fleet Management Framework using Model-Free Deep Reinforcement Learning and Change Point Detection [34.77250498401055]
  This paper introduces an adaptive model-free deep reinforcement approach that can recognize and adapt to the diurnal patterns in the ride-sharing environment with car-pooling. In addition to the adaptation logic in dispatching, this paper also proposes a dynamic, demand-aware vehicle-passenger matching and route planning framework.
  arXiv  Detail & Related papers  (2021-04-01T02:14:01Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)
• Meta-Reinforcement Learning for Trajectory Design in Wireless UAV Networks [151.65541208130995]
  A drone base station (DBS) is dispatched to provide uplink connectivity to ground users whose demand is dynamic and unpredictable. In this case, the DBS's trajectory must be adaptively adjusted to satisfy the dynamic user access requests. A meta-learning algorithm is proposed in order to adapt the DBS's trajectory when it encounters novel environments.
  arXiv  Detail & Related papers  (2020-05-25T20:43:59Z)
• IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control [4.273991039651846]
  Scaling adaptive traffic-signal control involves dealing with state and action spaces. We introduce Inductive Graph Reinforcement Learning (IG-RL) based on graph-convolutional networks. Our model can generalize to new road networks, traffic distributions, and traffic regimes.
  arXiv  Detail & Related papers  (2020-03-06T17:17:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.