Mitigating Partial Observability in Adaptive Traffic Signal Control with Transformers

• URL: http://arxiv.org/abs/2409.10693v1
• Date: Mon, 16 Sep 2024 19:46:15 GMT
• Title: Mitigating Partial Observability in Adaptive Traffic Signal Control with Transformers
• Authors: Xiaoyu Wang, Ayal Taitler, Scott Sanner, Baher Abdulhai,
• Abstract summary: Reinforcement Learning (RL) has emerged as a promising approach to enhancing adaptive traffic signal control (ATSC) systems. This paper presents the integration of Transformer-based controllers into ATSC systems to address partial observability (PO) The results showcase the Transformer-based model's ability to capture significant information from historical observations, leading to better control policies and improved traffic flow.
• Score: 26.1987660654434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Efficient traffic signal control is essential for managing urban transportation, minimizing congestion, and improving safety and sustainability. Reinforcement Learning (RL) has emerged as a promising approach to enhancing adaptive traffic signal control (ATSC) systems, allowing controllers to learn optimal policies through interaction with the environment. However, challenges arise due to partial observability (PO) in traffic networks, where agents have limited visibility, hindering effectiveness. This paper presents the integration of Transformer-based controllers into ATSC systems to address PO effectively. We propose strategies to enhance training efficiency and effectiveness, demonstrating improved coordination capabilities in real-world scenarios. The results showcase the Transformer-based model's ability to capture significant information from historical observations, leading to better control policies and improved traffic flow. This study highlights the potential of leveraging the advanced Transformer architecture to enhance urban transportation management.

Related papers

• GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching [82.19172267487998]
  GARLIC: a framework of GPT-Augmented Reinforcement Learning with Intelligent Control for vehicle dispatching. This paper introduces GARLIC: a framework of GPT-Augmented Reinforcement Learning with Intelligent Control for vehicle dispatching.
  arXiv  Detail & Related papers  (2024-08-19T08:23:38Z)
• MoveLight: Enhancing Traffic Signal Control through Movement-Centric Deep Reinforcement Learning [13.369840354712021]
  MoveLight is a novel traffic signal control system that enhances urban traffic management through movement-centric deep reinforcement learning. By leveraging detailed real-time data and advanced machine learning techniques, MoveLight overcomes the limitations of traditional traffic signal control methods.
  arXiv  Detail & Related papers  (2024-07-24T14:17:16Z)
• Communication-Aware Reinforcement Learning for Cooperative Adaptive Cruise Control [15.31488551912888]
  Reinforcement Learning (RL) has proven effective in optimizing complex decision-making processes in CACC. MARL has shown remarkable potential by enabling coordinated actions among multiple CAVs. We propose Communication-Aware Reinforcement Learning (CA-RL) to address these challenges.
  arXiv  Detail & Related papers  (2024-07-12T03:28:24Z)
• Agent-Agnostic Centralized Training for Decentralized Multi-Agent Cooperative Driving [17.659812774579756]
  We propose an asymmetric actor-critic model that learns decentralized cooperative driving policies for autonomous vehicles. By employing attention neural networks with masking, our approach efficiently manages real-world traffic dynamics and partial observability.
  arXiv  Detail & Related papers  (2024-03-18T16:13:02Z)
• A Holistic Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation [53.39174966020085]
  Traffic signal control (TSC) is crucial for reducing traffic congestion that leads to smoother traffic flow, reduced idling time, and mitigated CO2 emissions. In this study, we explore the computer vision approach for TSC that modulates on-road traffic flows through visual observation. We introduce a holistic traffic simulation framework called TrafficDojo towards vision-based TSC and its benchmarking.
  arXiv  Detail & Related papers  (2024-03-11T16:42:29Z)
• DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback [109.84667902348498]
  Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network. Most prior TSC methods leverage deep reinforcement learning to search for a control policy. We propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness.
  arXiv  Detail & Related papers  (2023-06-13T05:58:57Z)
• SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control [7.387226437589183]
  SocialLight is a new multi-agent reinforcement learning method for traffic signal control. It learns cooperative traffic control policies by estimating the individual marginal contribution of agents on their local neighborhood. We benchmark our trained network against state-of-the-art traffic signal control methods on standard benchmarks in two traffic simulators.
  arXiv  Detail & Related papers  (2023-04-20T12:41:25Z)
• Learning energy-efficient driving behaviors by imitating experts [75.12960180185105]
  This paper examines the role of imitation learning in bridging the gap between control strategies and realistic limitations in communication and sensing. We show that imitation learning can succeed in deriving policies that, if adopted by 5% of vehicles, may boost the energy-efficiency of networks with varying traffic conditions by 15% using only local observations.
  arXiv  Detail & Related papers  (2022-06-28T17:08:31Z)
• AI-aided Traffic Control Scheme for M2M Communications in the Internet of Vehicles [61.21359293642559]
  The dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies. We consider a hybrid traffic control scheme and use proximal policy optimization (PPO) method to tackle it.
  arXiv  Detail & Related papers  (2022-03-05T10:54:05Z)
• Network-wide traffic signal control optimization using a multi-agent deep reinforcement learning [20.385286762476436]
  Inefficient traffic control may cause numerous problems such as traffic congestion and energy waste. This paper proposes a novel multi-agent reinforcement learning method, named KS-DDPG, to achieve optimal control by enhancing the cooperation between traffic signals.
  arXiv  Detail & Related papers  (2021-04-20T12:53:08Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.