Towards Autonomous Railway Operations: A Semi-Hierarchical Deep Reinforcement Learning Approach to the Vehicle Rescheduling Problem

• URL: http://arxiv.org/abs/2605.10257v1
• Date: Mon, 11 May 2026 09:23:50 GMT
• Title: Towards Autonomous Railway Operations: A Semi-Hierarchical Deep Reinforcement Learning Approach to the Vehicle Rescheduling Problem
• Abstract summary: Managing disruptions in railway traffic management is a major challenge.<n>Rising traffic density and infrastructure limits increase complexity.<n>Most dispatching still relies on human expertise due to the problem's exponential complexity.<n>This paper introduces a semi-hierarchical RL formulation tailored to operational constraints.
• Score: 1.0348904561315861
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Managing disruptions in railway traffic management is a major challenge. Rising traffic density and infrastructure limits increase complexity, making the Vehicle Routing and Scheduling Problem (VRSP) difficult to solve reliably and in real time. While Operational Research (OR) methods are widely used, most dispatching still relies on human expertise due to the problem's exponential combinatorial complexity. Reinforcement Learning (RL) has gained attention for its potential in multi-agent coordination, but existing RL approaches often underperform OR methods and struggle to scale in dense rail networks. This paper addresses this gap from a machine learning perspective by introducing a semi-hierarchical RL formulation tailored to operational railway constraints. The method separates dispatching from routing through dedicated action and observation spaces, enabling policies to specialise in distinct decision scopes and addressing the imbalance between rare dispatch decisions and frequent routing updates. The approach is evaluated on the Flatland-RL simulator across five difficulty levels and 50 random seeds, with 7 to 80 trains. Results show substantially improved coordination, resource utilisation, and robustness compared with heuristic baselines and monolithic RL, nearly doubling the number of trains reaching their destinations, while keeping deadlock rates below 5% and adaptively sequencing, delaying, or cancelling trains under heavy congestion.

Related papers

• HiRAD: A Flexible Large-Scale AGV Routing System [6.059667257410882]
  We propose HiRAD, a hierarchical RL framework for continuous-space AGV routing with real-time guarantees.<n>HiRAD reduces makespan by 45 percent to 63 percent and shortens end-to-end runtime.
  arXiv  Detail & Related papers  (2026-09-09T05:50:17Z)
• A Temporal Planning Framework for Disruption Aware Dynamic Route Optimization in Heterogeneous Railway Systems [0.0]
  This study proposes a framework based on temporal planning for dynamic route optimization and disruption management.<n>It generates conflict-free timestamped operational plans specifying both optimized schedules and executable action sequences.
  arXiv  Detail & Related papers  (2026-06-12T16:00:31Z)
• Multi-Agent DRL for V2X Resource Allocation: Disentangling Challenges and Benchmarking Solutions [51.22818149833102]
  Multi-agent reinforcement learning (MARL) has emerged as a promising approach for radio resource allocation in vehicle-to-everything (C-V2X) networks.<n>However, the multifaceted challenges inherent to MARL are often intertwined, making it difficult to understand their individual impact on performance in vehicular environments.<n>We bridge this gap by formulating C-V2X RRA as a sequence of multi-agent interference games with progressively increasing complexity.
  arXiv  Detail & Related papers  (2026-02-18T14:46:56Z)
• Human-Centric Traffic Signal Control for Equity: A Multi-Agent Action Branching Deep Reinforcement Learning Approach [5.2437780355984165]
  We propose MA2B-DDQN, a human-centric multi-agent action-branching double Deep Q-Network (DQN) framework.<n>Our key contribution is an action-branching discrete control formulation that decomposes corridor control into local, per-intersection actions.<n>We also design a human-centric reward that penalizes the number of delayed individuals in the corridor, accounting for pedestrians, vehicle occupants, and transit passengers.
  arXiv  Detail & Related papers  (2026-02-03T00:56:03Z)
• Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail [85.47497935739936]
  Alpamayo-R1 (AR1) is a vision-language-action model that integrates Chain of Causation reasoning with trajectory planning.<n>We show AR1 achieves 12% improvement in planning accuracy on challenging cases compared to a trajectory-only baseline.<n>We plan to release AR1 models and a subset of the CoC in a future update.
  arXiv  Detail & Related papers  (2025-10-30T01:25:34Z)
• An Agentic Framework with LLMs for Solving Complex Vehicle Routing Problems [66.60904891478687]
  We propose an Agentic Framework with LLMs (AFL) for solving complex vehicle routing problems.<n>AFL directly extracts knowledge from raw inputs and enables self-contained code generation.<n>We show that AFL substantially outperforms existing LLM-based baselines in both code reliability and solution feasibility.
  arXiv  Detail & Related papers  (2025-10-19T03:59:25Z)
• Accelerating Vehicle Routing via AI-Initialized Genetic Algorithms [53.75036695728983]
  Vehicle Routing Problems (VRP) are a fundamental NP-hard challenge in Evolutionary optimization.<n>We introduce an optimization framework where a reinforcement learning agent is trained on prior instances and quickly generates initial solutions.<n>This framework consistently outperforms current state-of-the-art solvers across various time budgets.
  arXiv  Detail & Related papers  (2025-04-08T15:21:01Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• Generalizing Cooperative Eco-driving via Multi-residual Task Learning [6.864745785996583]
  Multi-residual Task Learning (MRTL) is a generic learning framework based on multi-task learning. MRTL decomposes control into nominal components that are effectively solved by conventional control methods and residual terms. We employ MRTL for fleet-level emission reduction in mixed traffic using autonomous vehicles as a means of system control.
  arXiv  Detail & Related papers  (2024-03-07T05:25:34Z)
• Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning [1.6752182911522522]
  Frequent lane changes during congestion at freeway bottlenecks such as merge and weaving areas further reduce roadway capacity. The emergence of deep reinforcement learning (RL) and connected and automated vehicle technology provides a possible solution to improve mobility and energy efficiency at freeway bottlenecks through cooperative lane changing. In this study, a decentralized cooperative lane-changing controller was developed using a multi-agent deep RL paradigm. The results of this study show that cooperative lane changing enabled by multi-agent deep RL yields superior performance to human drivers in term of traffic throughput, vehicle speed, number of stops per vehicle, vehicle fuel efficiency, and emissions.
  arXiv  Detail & Related papers  (2021-10-05T18:29:13Z)
• Flatland Competition 2020: MAPF and MARL for Efficient Train Coordination on a Grid World [49.80905654161763]
  The Flatland competition aimed at finding novel approaches to solve the vehicle re-scheduling problem (VRSP) The VRSP is concerned with scheduling trips in traffic networks and the re-scheduling of vehicles when disruptions occur. The ever-growing complexity of modern railway networks makes dynamic real-time scheduling of traffic virtually impossible.
  arXiv  Detail & Related papers  (2021-03-30T17:13:29Z)
• Solving the single-track train scheduling problem via Deep Reinforcement Learning [0.0]
  Induced delays propagate through the network, leading to a mismatch in demand and offer for goods and passengers. It is the duty of human traffic controllers, the so-called dispatchers, to do their best to minimize the impact on traffic. This paper investigates Machine Learning-based methods for tackling this problem, proposing two different Deep Q-Learning methods.
  arXiv  Detail & Related papers  (2020-09-01T14:03:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.