DataLight: Offline Data-Driven Traffic Signal Control
- URL: http://arxiv.org/abs/2303.10828v2
- Date: Thu, 2 May 2024 10:43:03 GMT
- Title: DataLight: Offline Data-Driven Traffic Signal Control
- Authors: Liang Zhang, Yutong Zhang, Jianming Deng, Chen Li,
- Abstract summary: Reinforcement learning (RL) has emerged as a promising solution for addressing traffic signal control (TSC) challenges.
This study introduces an innovative offline data-driven approach, called DataLight.
DataLight employs effective state representations and reward function by capturing vehicular speed information.
- Score: 9.393196900855648
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reinforcement learning (RL) has emerged as a promising solution for addressing traffic signal control (TSC) challenges. While most RL-based TSC systems typically employ an online approach, facilitating frequent active interaction with the environment, learning such strategies in the real world is impractical due to safety and risk concerns. To tackle these challenges, this study introduces an innovative offline data-driven approach, called DataLight. DataLight employs effective state representations and reward function by capturing vehicular speed information within the environment. It then segments roads to capture spatial information and further enhances the spatially segmented state representations with sequential modeling. The experimental results demonstrate the effectiveness of DataLight, showcasing superior performance compared to both state-of-the-art online and offline TSC methods. Additionally, DataLight exhibits robust learning capabilities concerning real-world deployment issues. The code is available at https://github.com/LiangZhang1996/DataLight.
Related papers
- Traffic Signal Control Using Lightweight Transformers: An
Offline-to-Online RL Approach [6.907105812732423]
We propose DTLight, a lightweight Decision Transformer-based TSC method that can learn policy from easily accessible offline datasets.
DTLight pre-trained purely on offline datasets can outperform state-of-the-art online RL-based methods in most scenarios.
Experiment results also show that online fine-tuning further improves the performance of DTLight by up to 42.6% over the best online RL baseline methods.
arXiv Detail & Related papers (2023-12-12T23:21:57Z) - A Fully Data-Driven Approach for Realistic Traffic Signal Control Using
Offline Reinforcement Learning [18.2541182874636]
We propose a fully Data-Driven and simulator-free framework for realistic Traffic Signal Control (D2TSC)
We combine well-established traffic flow theory with machine learning to infer the reward signals from coarse-grained traffic data.
Our approach achieves superior performance over conventional and offline RL baselines, and also enjoys much better real-world applicability.
arXiv Detail & Related papers (2023-11-27T15:29:21Z) - Reinforcement Learning with Human Feedback for Realistic Traffic
Simulation [53.85002640149283]
Key element of effective simulation is the incorporation of realistic traffic models that align with human knowledge.
This study identifies two main challenges: capturing the nuances of human preferences on realism and the unification of diverse traffic simulation models.
arXiv Detail & Related papers (2023-09-01T19:29:53Z) - Learning to Control Autonomous Fleets from Observation via Offline
Reinforcement Learning [3.9121134770873733]
We propose to formalize the control of Autonomous Mobility-on-Demand systems through the lens of offline reinforcement learning.
We show that offline RL is a promising paradigm for the application of RL-based solutions within economically-critical systems.
arXiv Detail & Related papers (2023-02-28T18:31:07Z) - Challenges and Opportunities in Offline Reinforcement Learning from
Visual Observations [58.758928936316785]
offline reinforcement learning from visual observations with continuous action spaces remains under-explored.
We show that modifications to two popular vision-based online reinforcement learning algorithms suffice to outperform existing offline RL methods.
arXiv Detail & Related papers (2022-06-09T22:08:47Z) - TL-GAN: Improving Traffic Light Recognition via Data Synthesis for
Autonomous Driving [8.474436072102844]
We propose a novel traffic light generation approach TL-GAN to synthesize the data of rare classes to improve traffic light recognition for autonomous driving.
In the image synthesis stage, our approach enables conditional generation to allow full control of the color of the generated traffic light images.
In the sequence assembling stage, we design the style mixing and adaptive template to synthesize realistic and diverse traffic light sequences.
arXiv Detail & Related papers (2022-03-28T18:12:35Z) - ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal
Control [5.219291917441908]
This paper proposes a novel model-based meta-reinforcement learning framework (ModelLight) for traffic signal control.
Within ModelLight, an ensemble of models for road intersections and the optimization-based meta-learning method are used to improve the data efficiency of an RL-based traffic light control method.
Experiments on real-world datasets demonstrate that ModelLight can outperform state-of-the-art traffic light control algorithms.
arXiv Detail & Related papers (2021-11-15T20:25:08Z) - Offline Reinforcement Learning from Images with Latent Space Models [60.69745540036375]
offline reinforcement learning (RL) refers to the problem of learning policies from a static dataset of environment interactions.
We build on recent advances in model-based algorithms for offline RL, and extend them to high-dimensional visual observation spaces.
Our approach is both tractable in practice and corresponds to maximizing a lower bound of the ELBO in the unknown POMDP.
arXiv Detail & Related papers (2020-12-21T18:28:17Z) - Deep traffic light detection by overlaying synthetic context on
arbitrary natural images [49.592798832978296]
We propose a method to generate artificial traffic-related training data for deep traffic light detectors.
This data is generated using basic non-realistic computer graphics to blend fake traffic scenes on top of arbitrary image backgrounds.
It also tackles the intrinsic data imbalance problem in traffic light datasets, caused mainly by the low amount of samples of the yellow state.
arXiv Detail & Related papers (2020-11-07T19:57:22Z) - OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement
Learning [107.6943868812716]
In many practical applications, the situation is reversed: an agent may have access to large amounts of undirected offline experience data, while access to the online environment is severely limited.
Our main insight is that, when presented with offline data composed of a variety of behaviors, an effective way to leverage this data is to extract a continuous space of recurring and temporally extended primitive behaviors.
In addition to benefiting offline policy optimization, we show that performing offline primitive learning in this way can also be leveraged for improving few-shot imitation learning.
arXiv Detail & Related papers (2020-10-26T14:31:08Z) - AWAC: Accelerating Online Reinforcement Learning with Offline Datasets [84.94748183816547]
We show that our method, advantage weighted actor critic (AWAC), enables rapid learning of skills with a combination of prior demonstration data and online experience.
Our results show that incorporating prior data can reduce the time required to learn a range of robotic skills to practical time-scales.
arXiv Detail & Related papers (2020-06-16T17:54:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.