VLMLight: Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning

• URL: http://arxiv.org/abs/2505.19486v1
• Date: Mon, 26 May 2025 04:12:57 GMT
• Title: VLMLight: Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning
• Authors: Maonan Wang, Yirong Chen, Aoyu Pang, Yuxin Cai, Chung Shue Chen, Yuheng Kan, Man-On Pun,
• Abstract summary: VLMLight is a novel framework that integrates vision-language meta-control with dual-branch reasoning.<n>A large language model (LLM) serves as a safety-prioritized meta-controller, selecting between a fast RL policy for routine traffic and a structured reasoning branch for critical cases.<n> Experiments show that VLMLight reduces waiting times for emergency vehicles by up to 65% over RL-only systems.
• Score: 3.475835936400513
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Traffic signal control (TSC) is a core challenge in urban mobility, where real-time decisions must balance efficiency and safety. Existing methods - ranging from rule-based heuristics to reinforcement learning (RL) - often struggle to generalize to complex, dynamic, and safety-critical scenarios. We introduce VLMLight, a novel TSC framework that integrates vision-language meta-control with dual-branch reasoning. At the core of VLMLight is the first image-based traffic simulator that enables multi-view visual perception at intersections, allowing policies to reason over rich cues such as vehicle type, motion, and spatial density. A large language model (LLM) serves as a safety-prioritized meta-controller, selecting between a fast RL policy for routine traffic and a structured reasoning branch for critical cases. In the latter, multiple LLM agents collaborate to assess traffic phases, prioritize emergency vehicles, and verify rule compliance. Experiments show that VLMLight reduces waiting times for emergency vehicles by up to 65% over RL-only systems, while preserving real-time performance in standard conditions with less than 1% degradation. VLMLight offers a scalable, interpretable, and safety-aware solution for next-generation traffic signal control.

Related papers

• SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models [63.71984266104757]
  Multimodal Large Language Models (MLLMs) can process both visual and textual data.<n>We propose SafeAuto, a novel framework that enhances MLLM-based autonomous driving systems by incorporating both unstructured and structured knowledge.
  arXiv  Detail & Related papers  (2025-02-28T21:53:47Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• Large Language Models (LLMs) as Traffic Control Systems at Urban Intersections: A New Paradigm [5.233512464561313]
  This study introduces a novel approach for traffic control systems by using Large Language Models (LLMs) as traffic controllers. The study utilizes their logical reasoning, scene understanding, and decision-making capabilities to optimize throughput and provide feedback based on traffic conditions in real-time.
  arXiv  Detail & Related papers  (2024-11-16T19:23:52Z)
• LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environments [3.7788636451616697]
  This work introduces an innovative approach that integrates Large Language Models into traffic signal control systems. A hybrid framework that augments LLMs with a suite of perception and decision-making tools is proposed. The findings from our simulations attest to the system's adeptness in adjusting to a multiplicity of traffic environments.
  arXiv  Detail & Related papers  (2024-03-13T08:41:55Z)
• A Holistic Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation [53.39174966020085]
  Traffic signal control (TSC) is crucial for reducing traffic congestion that leads to smoother traffic flow, reduced idling time, and mitigated CO2 emissions. In this study, we explore the computer vision approach for TSC that modulates on-road traffic flows through visual observation. We introduce a holistic traffic simulation framework called TrafficDojo towards vision-based TSC and its benchmarking.
  arXiv  Detail & Related papers  (2024-03-11T16:42:29Z)
• LLMLight: Large Language Models as Traffic Signal Control Agents [25.438040499152745]
  Traffic Signal Control (TSC) is a crucial component in urban traffic management, aiming to optimize road network efficiency and reduce congestion.<n>This paper presents LLMLight, a novel framework employing Large Language Models (LLMs) as decision-making agents for TSC.
  arXiv  Detail & Related papers  (2023-12-26T13:17:06Z)
• DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback [109.84667902348498]
  Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network. Most prior TSC methods leverage deep reinforcement learning to search for a control policy. We propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness.
  arXiv  Detail & Related papers  (2023-06-13T05:58:57Z)
• EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System [4.622745478006317]
  Emergency vehicles (EMVs) play a crucial role in responding to time-critical calls such as medical emergencies and fire outbreaks in urban areas. Existing methods for EMV dispatch typically optimize routes based on historical traffic-flow data and design traffic signal pre-emption accordingly. We propose EMVLight, a decentralized reinforcement learning framework for joint dynamic EMV routing and traffic signal pre-emption.
  arXiv  Detail & Related papers  (2022-06-27T16:46:20Z)
• ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control [5.219291917441908]
  This paper proposes a novel model-based meta-reinforcement learning framework (ModelLight) for traffic signal control. Within ModelLight, an ensemble of models for road intersections and the optimization-based meta-learning method are used to improve the data efficiency of an RL-based traffic light control method. Experiments on real-world datasets demonstrate that ModelLight can outperform state-of-the-art traffic light control algorithms.
  arXiv  Detail & Related papers  (2021-11-15T20:25:08Z)
• Integrated Decision and Control at Multi-Lane Intersections with Mixed Traffic Flow [6.233422723925688]
  This paper develops a learning-based algorithm to deal with complex intersections with mixed traffic flows. We first consider different velocity models for green and red lights in the training process and use a finite state machine to handle different modes of light transformation. Then we design different types of distance constraints for vehicles, traffic lights, pedestrians, bicycles respectively and formulize the constrained optimal control problems.
  arXiv  Detail & Related papers  (2021-08-30T07:55:32Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.