SocialLight: Distributed Cooperation Learning towards Network-Wide
Traffic Signal Control
- URL: http://arxiv.org/abs/2305.16145v1
- Date: Thu, 20 Apr 2023 12:41:25 GMT
- Title: SocialLight: Distributed Cooperation Learning towards Network-Wide
Traffic Signal Control
- Authors: Harsh Goel, Yifeng Zhang, Mehul Damani, and Guillaume Sartoretti
- Abstract summary: SocialLight is a new multi-agent reinforcement learning method for traffic signal control.
It learns cooperative traffic control policies by estimating the individual marginal contribution of agents on their local neighborhood.
We benchmark our trained network against state-of-the-art traffic signal control methods on standard benchmarks in two traffic simulators.
- Score: 7.387226437589183
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Many recent works have turned to multi-agent reinforcement learning (MARL)
for adaptive traffic signal control to optimize the travel time of vehicles
over large urban networks. However, achieving effective and scalable
cooperation among junctions (agents) remains an open challenge, as existing
methods often rely on extensive, non-generalizable reward shaping or on
non-scalable centralized learning. To address these problems, we propose a new
MARL method for traffic signal control, SocialLight, which learns cooperative
traffic control policies by distributedly estimating the individual marginal
contribution of agents on their local neighborhood. SocialLight relies on the
Asynchronous Actor Critic (A3C) framework, and makes learning scalable by
learning a locally-centralized critic conditioned over the states and actions
of neighboring agents, used by agents to estimate individual contributions by
counterfactual reasoning. We further introduce important modifications to the
advantage calculation that help stabilize policy updates. These modifications
decouple the impact of the neighbors' actions on the computed advantages,
thereby reducing the variance in the gradient updates. We benchmark our trained
network against state-of-the-art traffic signal control methods on standard
benchmarks in two traffic simulators, SUMO and CityFlow. Our results show that
SocialLight exhibits improved scalability to larger road networks and better
performance across usual traffic metrics.
Related papers
- Single-Agent Actor Critic for Decentralized Cooperative Driving [17.659812774579756]
We introduce a novel asymmetric actor-critic model aimed at learning decentralized cooperative driving policies for autonomous vehicles.
Our approach employs attention neural networks with masking to handle the dynamic nature of real-world traffic flow and partial observability.
arXiv Detail & Related papers (2024-03-18T16:13:02Z) - A Holistic Framework Towards Vision-based Traffic Signal Control with
Microscopic Simulation [53.39174966020085]
Traffic signal control (TSC) is crucial for reducing traffic congestion that leads to smoother traffic flow, reduced idling time, and mitigated CO2 emissions.
In this study, we explore the computer vision approach for TSC that modulates on-road traffic flows through visual observation.
We introduce a holistic traffic simulation framework called TrafficDojo towards vision-based TSC and its benchmarking.
arXiv Detail & Related papers (2024-03-11T16:42:29Z) - DenseLight: Efficient Control for Large-scale Traffic Signals with Dense
Feedback [109.84667902348498]
Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network.
Most prior TSC methods leverage deep reinforcement learning to search for a control policy.
We propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness.
arXiv Detail & Related papers (2023-06-13T05:58:57Z) - Cooperative Reinforcement Learning on Traffic Signal Control [3.759936323189418]
Traffic signal control is a challenging real-world problem aiming to minimize overall travel time by coordinating vehicle movements at road intersections.
Existing traffic signal control systems in use still rely heavily on oversimplified information and rule-based methods.
This paper proposes a cooperative, multi-objective architecture with age-decaying weights to better estimate multiple reward terms for traffic signal control optimization.
arXiv Detail & Related papers (2022-05-23T13:25:15Z) - AI-aided Traffic Control Scheme for M2M Communications in the Internet
of Vehicles [61.21359293642559]
The dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies.
We consider a hybrid traffic control scheme and use proximal policy optimization (PPO) method to tackle it.
arXiv Detail & Related papers (2022-03-05T10:54:05Z) - Learning to Help Emergency Vehicles Arrive Faster: A Cooperative
Vehicle-Road Scheduling Approach [24.505687255063986]
Vehicle-centric scheduling approaches recommend optimal paths for emergency vehicles.
Road-centric scheduling approaches aim to improve the traffic condition and assign a higher priority for EVs to pass an intersection.
We propose LEVID, a cooperative VehIcle-roaD scheduling approach including a real-time route planning module and a collaborative traffic signal control module.
arXiv Detail & Related papers (2022-02-20T10:25:15Z) - A Deep Reinforcement Learning Approach for Traffic Signal Control
Optimization [14.455497228170646]
Inefficient traffic signal control methods may cause numerous problems, such as traffic congestion and waste of energy.
This paper first proposes a multi-agent deep deterministic policy gradient (MADDPG) method by extending the actor-critic policy gradient algorithms.
arXiv Detail & Related papers (2021-07-13T14:11:04Z) - Courteous Behavior of Automated Vehicles at Unsignalized Intersections
via Reinforcement Learning [30.00761722505295]
We propose a novel approach to optimize traffic flow at intersections in mixed traffic situations using deep reinforcement learning.
Our reinforcement learning agent learns a policy for a centralized controller to let connected autonomous vehicles at unsignalized intersections give up their right of way and yield to other vehicles to optimize traffic flow.
arXiv Detail & Related papers (2021-06-11T13:16:48Z) - End-to-End Intersection Handling using Multi-Agent Deep Reinforcement
Learning [63.56464608571663]
Navigating through intersections is one of the main challenging tasks for an autonomous vehicle.
In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided.
We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
arXiv Detail & Related papers (2021-04-28T07:54:40Z) - MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city.
Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent.
We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
arXiv Detail & Related papers (2021-01-04T03:06:08Z) - Optimizing Mixed Autonomy Traffic Flow With Decentralized Autonomous
Vehicles and Multi-Agent RL [63.52264764099532]
We study the ability of autonomous vehicles to improve the throughput of a bottleneck using a fully decentralized control scheme in a mixed autonomy setting.
We apply multi-agent reinforcement algorithms to this problem and demonstrate that significant improvements in bottleneck throughput, from 20% at a 5% penetration rate to 33% at a 40% penetration rate, can be achieved.
arXiv Detail & Related papers (2020-10-30T22:06:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.