Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement
Learning
- URL: http://arxiv.org/abs/2105.00376v1
- Date: Sun, 2 May 2021 02:08:07 GMT
- Title: Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement
Learning
- Authors: Jiawei Wang and Lijun Sun
- Abstract summary: Bus bunching is a common phenomenon that undermines the reliability and efficiency of bus services.
We formulate route-level bus fleet control as an asynchronous multi-agent reinforcement learning problem.
We extend the classical actor-critic architecture to handle the asynchronous issue.
- Score: 11.168121941015013
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The bus system is a critical component of sustainable urban transportation.
However, due to the significant uncertainties in passenger demand and traffic
conditions, bus operation is unstable in nature and bus bunching has become a
common phenomenon that undermines the reliability and efficiency of bus
services. Despite recent advances in multi-agent reinforcement learning (MARL)
on traffic control, little research has focused on bus fleet control due to the
tricky asynchronous characteristic -- control action only happens when a bus
arrives at a bus stop and thus agents do not act simultaneously. In this study,
we formulate route-level bus fleet control as an asynchronous multi-agent
reinforcement learning (ASMR) problem and extend the classical actor-critic
architecture to handle the asynchronous issue. Specifically, we design a novel
critic network to effectively approximate the marginal contribution for other
agents, in which graph attention neural network is used to conduct inductive
learning for policy evaluation. The critic structure also helps the ego agent
optimize its policy more efficiently. We evaluate the proposed framework on
real-world bus services and actual passenger demand derived from smart card
data. Our results show that the proposed model outperforms both traditional
headway-based control methods and existing MARL methods.
Related papers
- Single-Agent Actor Critic for Decentralized Cooperative Driving [17.659812774579756]
We introduce a novel asymmetric actor-critic model aimed at learning decentralized cooperative driving policies for autonomous vehicles.
Our approach employs attention neural networks with masking to handle the dynamic nature of real-world traffic flow and partial observability.
arXiv Detail & Related papers (2024-03-18T16:13:02Z) - RL-MSA: a Reinforcement Learning-based Multi-line bus Scheduling
Approach [0.0]
Existing approaches typically generate a bus scheduling scheme in an offline manner and then schedule buses according to the scheme.
In this paper, MLBSP is modeled as a Markov Decision Process (MDP)
A Reinforcement Learning-based Multi-line bus Scheduling Approach (RL-MSA) is proposed for bus scheduling at both the offline and online phases.
arXiv Detail & Related papers (2024-03-11T07:07:05Z) - An Online Approach to Solving Public Transit Stationing and Dispatch
Problem [7.948662269574215]
Transit agencies keep a limited number of vehicles in reserve and dispatch them to relieve the affected routes during disruptions.
This paper describes a principled approach using non-myopic sequential decision procedures to solve the problem.
Our experiments show that the proposed framework serves 2% more passengers while reducing deadhead miles by 40%.
arXiv Detail & Related papers (2024-03-05T21:48:29Z) - Interactive Autonomous Navigation with Internal State Inference and
Interactivity Estimation [58.21683603243387]
We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework.
These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents.
Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
arXiv Detail & Related papers (2023-11-27T18:57:42Z) - Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning [48.667697255912614]
Mean-field reinforcement learning addresses the policy of a representative agent interacting with the infinite population of identical agents.
We propose Safe-M$3$-UCRL, the first model-based mean-field reinforcement learning algorithm that attains safe policies even in the case of unknown transitions.
Our algorithm effectively meets the demand in critical areas while ensuring service accessibility in regions with low demand.
arXiv Detail & Related papers (2023-06-29T15:57:07Z) - Learning to Sail Dynamic Networks: The MARLIN Reinforcement Learning
Framework for Congestion Control in Tactical Environments [53.08686495706487]
This paper proposes an RL framework that leverages an accurate and parallelizable emulation environment to reenact the conditions of a tactical network.
We evaluate our RL learning framework by training a MARLIN agent in conditions replicating a bottleneck link transition between a Satellite Communication (SATCOM) and an UHF Wide Band (UHF) radio link.
arXiv Detail & Related papers (2023-06-27T16:15:15Z) - Model-Based Reinforcement Learning with Isolated Imaginations [61.67183143982074]
We propose Iso-Dream++, a model-based reinforcement learning approach.
We perform policy optimization based on the decoupled latent imaginations.
This enables long-horizon visuomotor control tasks to benefit from isolating mixed dynamics sources in the wild.
arXiv Detail & Related papers (2023-03-27T02:55:56Z) - Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement
Learning [9.48157144651867]
We present a novel approach for Robust on-ramp merge of CAVs via Augmented and Multi-modal Reinforcement Learning.
Specifically, we formulate the on-ramp merging problem as a Markov decision process (MDP) by taking driving safety, comfort driving behavior, and traffic efficiency into account.
To provide reliable merging maneuvers, we simultaneously leverage BSM and surveillance images for multi-modal observation.
arXiv Detail & Related papers (2022-07-21T16:34:57Z) - Robust Dynamic Bus Control: A Distributional Multi-agent Reinforcement
Learning Approach [11.168121941015013]
Bus bunching is a common phenomenon that undermines the efficiency and reliability of bus systems.
We develop a distributional MARL framework -- IQNC-M -- to learn continuous control.
Our results show that the proposed IQNC-M framework can effectively handle the various extreme events.
arXiv Detail & Related papers (2021-11-02T23:41:09Z) - End-to-End Intersection Handling using Multi-Agent Deep Reinforcement
Learning [63.56464608571663]
Navigating through intersections is one of the main challenging tasks for an autonomous vehicle.
In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided.
We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
arXiv Detail & Related papers (2021-04-28T07:54:40Z) - Reinforcement Learning for Autonomous Driving with Latent State
Inference and Spatial-Temporal Relationships [46.965260791099986]
We show that explicitly inferring the latent state and encoding spatial-temporal relationships in a reinforcement learning framework can help address this difficulty.
We encode prior knowledge on the latent states of other drivers through a framework that combines the reinforcement learner with a supervised learner.
The proposed framework significantly improves performance in the context of navigating T-intersections compared with state-of-the-art baseline approaches.
arXiv Detail & Related papers (2020-11-09T08:55:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.