Robust Dynamic Bus Control: A Distributional Multi-agent Reinforcement Learning Approach

• URL: http://arxiv.org/abs/2111.01946v1
• Date: Tue, 2 Nov 2021 23:41:09 GMT
• Title: Robust Dynamic Bus Control: A Distributional Multi-agent Reinforcement Learning Approach
• Authors: Jiawei Wang, Lijun Sun
• Abstract summary: Bus bunching is a common phenomenon that undermines the efficiency and reliability of bus systems. We develop a distributional MARL framework -- IQNC-M -- to learn continuous control. Our results show that the proposed IQNC-M framework can effectively handle the various extreme events.
• Score: 11.168121941015013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bus system is a critical component of sustainable urban transportation. However, the operation of a bus fleet is unstable in nature, and bus bunching has become a common phenomenon that undermines the efficiency and reliability of bus systems. Recently research has demonstrated the promising application of multi-agent reinforcement learning (MARL) to achieve efficient vehicle holding control to avoid bus bunching. However, existing studies essentially overlook the robustness issue resulting from various events, perturbations and anomalies in a transit system, which is of utmost importance when transferring the models for real-world deployment/application. In this study, we integrate implicit quantile network and meta-learning to develop a distributional MARL framework -- IQNC-M -- to learn continuous control. The proposed IQNC-M framework achieves efficient and reliable control decisions through better handling various uncertainties/events in real-time transit operations. Specifically, we introduce an interpretable meta-learning module to incorporate global information into the distributional MARL framework, which is an effective solution to circumvent the credit assignment issue in the transit system. In addition, we design a specific learning procedure to train each agent within the framework to pursue a robust control policy. We develop simulation environments based on real-world bus services and passenger demand data and evaluate the proposed framework against both traditional holding control models and state-of-the-art MARL models. Our results show that the proposed IQNC-M framework can effectively handle the various extreme events, such as traffic state perturbations, service interruptions, and demand surges, thus improving both efficiency and reliability of the system.

Related papers

• MetaTrading: An Immersion-Aware Model Trading Framework for Vehicular Metaverse Services [94.61039892220037]
  We present a novel immersion-aware model trading framework that incentivizes metaverse users (MUs) to contribute learning models for augmented reality (AR) services in the vehicular metaverse. Considering dynamic network conditions and privacy concerns, we formulate the reward decisions of MSPs as a multi-agent Markov decision process. Experimental results demonstrate that the proposed framework can effectively provide higher-value models for object detection and classification in AR services on real AR-related vehicle datasets.
  arXiv  Detail & Related papers  (2024-10-25T16:20:46Z)
• Agent-Agnostic Centralized Training for Decentralized Multi-Agent Cooperative Driving [17.659812774579756]
  We propose an asymmetric actor-critic model that learns decentralized cooperative driving policies for autonomous vehicles. By employing attention neural networks with masking, our approach efficiently manages real-world traffic dynamics and partial observability.
  arXiv  Detail & Related papers  (2024-03-18T16:13:02Z)
• LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environments [3.7788636451616697]
  This work introduces an innovative approach that integrates Large Language Models into traffic signal control systems. A hybrid framework that augments LLMs with a suite of perception and decision-making tools is proposed. The findings from our simulations attest to the system's adeptness in adjusting to a multiplicity of traffic environments.
  arXiv  Detail & Related papers  (2024-03-13T08:41:55Z)
• MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying [110.83590008788745]
  Motion prediction is crucial for autonomous driving systems to understand complex driving scenarios and make informed decisions. In this paper, we propose Motion TRansformer (MTR) frameworks to address these challenges. The initial MTR framework utilizes a transformer encoder-decoder structure with learnable intention queries. We introduce an advanced MTR++ framework, extending the capability of MTR to simultaneously predict multimodal motion for multiple agents.
  arXiv  Detail & Related papers  (2023-06-30T16:23:04Z)
• Model-Based Reinforcement Learning with Isolated Imaginations [61.67183143982074]
  We propose Iso-Dream++, a model-based reinforcement learning approach. We perform policy optimization based on the decoupled latent imaginations. This enables long-horizon visuomotor control tasks to benefit from isolating mixed dynamics sources in the wild.
  arXiv  Detail & Related papers  (2023-03-27T02:55:56Z)
• MMRNet: Improving Reliability for Multimodal Object Detection and Segmentation for Bin Picking via Multimodal Redundancy [68.7563053122698]
  We propose a reliable object detection and segmentation system with MultiModal Redundancy (MMRNet) This is the first system that introduces the concept of multimodal redundancy to address sensor failure issues during deployment. We present a new label-free multi-modal consistency (MC) score that utilizes the output from all modalities to measure the overall system output reliability and uncertainty.
  arXiv  Detail & Related papers  (2022-10-19T19:15:07Z)
• Isolating and Leveraging Controllable and Noncontrollable Visual Dynamics in World Models [65.97707691164558]
  We present Iso-Dream, which improves the Dream-to-Control framework in two aspects. First, by optimizing inverse dynamics, we encourage world model to learn controllable and noncontrollable sources. Second, we optimize the behavior of the agent on the decoupled latent imaginations of the world model.
  arXiv  Detail & Related papers  (2022-05-27T08:07:39Z)
• UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning [1.1339580074756188]
  Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data. Self-driving vehicles (SDV) learn a policy, which potentially even outperforms the behavior in the sub-optimal data set. This motivates the use of model-based offline RL approaches, which leverage planning.
  arXiv  Detail & Related papers  (2021-11-22T10:37:52Z)
• A Modular and Transferable Reinforcement Learning Framework for the Fleet Rebalancing Problem [2.299872239734834]
  We propose a modular framework for fleet rebalancing based on model-free reinforcement learning (RL) We formulate RL state and action spaces as distributions over a grid of the operating area, making the framework scalable. Numerical experiments, using real-world trip and network data, demonstrate that this approach has several distinct advantages over baseline methods.
  arXiv  Detail & Related papers  (2021-05-27T16:32:28Z)
• Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning [11.168121941015013]
  Bus bunching is a common phenomenon that undermines the reliability and efficiency of bus services. We formulate route-level bus fleet control as an asynchronous multi-agent reinforcement learning problem. We extend the classical actor-critic architecture to handle the asynchronous issue.
  arXiv  Detail & Related papers  (2021-05-02T02:08:07Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.