Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2105.00376v1
• Date: Sun, 2 May 2021 02:08:07 GMT
• Title: Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning
• Authors: Jiawei Wang and Lijun Sun
• Abstract summary: Bus bunching is a common phenomenon that undermines the reliability and efficiency of bus services. We formulate route-level bus fleet control as an asynchronous multi-agent reinforcement learning problem. We extend the classical actor-critic architecture to handle the asynchronous issue.
• Score: 11.168121941015013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The bus system is a critical component of sustainable urban transportation. However, due to the significant uncertainties in passenger demand and traffic conditions, bus operation is unstable in nature and bus bunching has become a common phenomenon that undermines the reliability and efficiency of bus services. Despite recent advances in multi-agent reinforcement learning (MARL) on traffic control, little research has focused on bus fleet control due to the tricky asynchronous characteristic -- control action only happens when a bus arrives at a bus stop and thus agents do not act simultaneously. In this study, we formulate route-level bus fleet control as an asynchronous multi-agent reinforcement learning (ASMR) problem and extend the classical actor-critic architecture to handle the asynchronous issue. Specifically, we design a novel critic network to effectively approximate the marginal contribution for other agents, in which graph attention neural network is used to conduct inductive learning for policy evaluation. The critic structure also helps the ego agent optimize its policy more efficiently. We evaluate the proposed framework on real-world bus services and actual passenger demand derived from smart card data. Our results show that the proposed model outperforms both traditional headway-based control methods and existing MARL methods.

Related papers

• Improving Traffic Flow Predictions with SGCN-LSTM: A Hybrid Model for Spatial and Temporal Dependencies [55.2480439325792]
  This paper introduces the Signal-Enhanced Graph Convolutional Network Long Short Term Memory (SGCN-LSTM) model for predicting traffic speeds across road networks. Experiments on the PEMS-BAY road network traffic dataset demonstrate the SGCN-LSTM model's effectiveness.
  arXiv  Detail & Related papers  (2024-11-01T00:37:00Z)
• Semantic Communication for Cooperative Perception using HARQ [51.148203799109304]
  We leverage an importance map to distill critical semantic information, introducing a cooperative perception semantic communication framework. To counter the challenges posed by time-varying multipath fading, our approach incorporates the use of frequency-division multiplexing (OFDM) along with channel estimation and equalization strategies. We introduce a novel semantic error detection method that is integrated with our semantic communication framework in the spirit of hybrid automatic repeated request (HARQ)
  arXiv  Detail & Related papers  (2024-08-29T08:53:26Z)
• GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching [82.19172267487998]
  GARLIC: a framework of GPT-Augmented Reinforcement Learning with Intelligent Control for vehicle dispatching. This paper introduces GARLIC: a framework of GPT-Augmented Reinforcement Learning with Intelligent Control for vehicle dispatching.
  arXiv  Detail & Related papers  (2024-08-19T08:23:38Z)
• Agent-Agnostic Centralized Training for Decentralized Multi-Agent Cooperative Driving [17.659812774579756]
  We propose an asymmetric actor-critic model that learns decentralized cooperative driving policies for autonomous vehicles. By employing attention neural networks with masking, our approach efficiently manages real-world traffic dynamics and partial observability.
  arXiv  Detail & Related papers  (2024-03-18T16:13:02Z)
• RL-MSA: a Reinforcement Learning-based Multi-line bus Scheduling Approach [0.0]
  Existing approaches typically generate a bus scheduling scheme in an offline manner and then schedule buses according to the scheme. In this paper, MLBSP is modeled as a Markov Decision Process (MDP) A Reinforcement Learning-based Multi-line bus Scheduling Approach (RL-MSA) is proposed for bus scheduling at both the offline and online phases.
  arXiv  Detail & Related papers  (2024-03-11T07:07:05Z)
• An Online Approach to Solving Public Transit Stationing and Dispatch Problem [7.948662269574215]
  Transit agencies keep a limited number of vehicles in reserve and dispatch them to relieve the affected routes during disruptions. This paper describes a principled approach using non-myopic sequential decision procedures to solve the problem. Our experiments show that the proposed framework serves 2% more passengers while reducing deadhead miles by 40%.
  arXiv  Detail & Related papers  (2024-03-05T21:48:29Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning [48.667697255912614]
  Mean-field reinforcement learning addresses the policy of a representative agent interacting with the infinite population of identical agents. We propose Safe-M$3$-UCRL, the first model-based mean-field reinforcement learning algorithm that attains safe policies even in the case of unknown transitions. Our algorithm effectively meets the demand in critical areas while ensuring service accessibility in regions with low demand.
  arXiv  Detail & Related papers  (2023-06-29T15:57:07Z)
• Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning [9.48157144651867]
  We present a novel approach for Robust on-ramp merge of CAVs via Augmented and Multi-modal Reinforcement Learning. Specifically, we formulate the on-ramp merging problem as a Markov decision process (MDP) by taking driving safety, comfort driving behavior, and traffic efficiency into account. To provide reliable merging maneuvers, we simultaneously leverage BSM and surveillance images for multi-modal observation.
  arXiv  Detail & Related papers  (2022-07-21T16:34:57Z)
• Robust Dynamic Bus Control: A Distributional Multi-agent Reinforcement Learning Approach [11.168121941015013]
  Bus bunching is a common phenomenon that undermines the efficiency and reliability of bus systems. We develop a distributional MARL framework -- IQNC-M -- to learn continuous control. Our results show that the proposed IQNC-M framework can effectively handle the various extreme events.
  arXiv  Detail & Related papers  (2021-11-02T23:41:09Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.