Online Action-Stacking Improves Reinforcement Learning Performance for Air Traffic Control

• URL: http://arxiv.org/abs/2601.04287v1
• Date: Wed, 07 Jan 2026 14:28:37 GMT
• Title: Online Action-Stacking Improves Reinforcement Learning Performance for Air Traffic Control
• Abstract summary: Online action-stacking is an inference-time wrapper for reinforcement learning policies.<n>It produces realistic air traffic control commands while allowing training on a much smaller discrete action space.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce online action-stacking, an inference-time wrapper for reinforcement learning policies that produces realistic air traffic control commands while allowing training on a much smaller discrete action space. Policies are trained with simple incremental heading or level adjustments, together with an action-damping penalty that reduces instruction frequency and leads agents to issue commands in short bursts. At inference, online action-stacking compiles these bursts of primitive actions into domain-appropriate compound clearances. Using Proximal Policy Optimisation and the BluebirdDT digital twin platform, we train agents to navigate aircraft along lateral routes, manage climb and descent to target flight levels, and perform two-aircraft collision avoidance under a minimum separation constraint. In our lateral navigation experiments, action stacking greatly reduces the number of issued instructions relative to a damped baseline and achieves comparable performance to a policy trained with a 37-dimensional action space, despite operating with only five actions. These results indicate that online action-stacking helps bridge a key gap between standard reinforcement learning formulations and operational ATC requirements, and provides a simple mechanism for scaling to more complex control scenarios.

Related papers

• Mind Your Steps: A General Learning Framework for Accurate Humanoid Foothold Tracking [78.30338456206984]
  We introduce a novel framework for training general-purpose 3D foothold-tracking policies.<n>By dynamically providing footstep support through a goal sampler, this method enables the learned policy to be agnostic to specific terrains.<n>Our new target representation effectively mitigates challenges arising in the real world, such as noisy and inaccurate pose estimation and foot contact estimation.
  arXiv  Detail & Related papers  (2026-06-06T16:44:56Z)
• Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control [50.28263951510334]
  We propose a hybrid hierarchical control framework for reactive dexterous grasping.<n>The proposed framework explicitly decouples high-level spatial intent from low-level joint execution.<n>We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline.
  arXiv  Detail & Related papers  (2026-05-05T04:49:38Z)
• Hypernetwork-Conditioned Reinforcement Learning for Robust Control of Fixed-Wing Aircraft under Actuator Failures [0.8164433158925594]
  This paper presents a reinforcement learning-based path-following controller for a small uncrewed aircraft system.<n>We consider parameter-efficient formulations based on Feature-wise Linear Modulation (FiLM) and Low-Rank Adaptation (LoRA)<n>We demonstrate that hypernetwork-conditioned policies can improve robustness compared to standard multilayer perceptron policies.
  arXiv  Detail & Related papers  (2026-04-03T18:50:31Z)
• Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration [58.4036440289082]
  Hand-object motion-capture (MoCap) offer large-scale, contact-rich demonstrations and hold promise for dexterous robotic scopes.<n>We introduce Dexplore, a unified single-loop optimization that performs repositories and tracking to learn robot control policies directly from MoCap at scale.
  arXiv  Detail & Related papers  (2025-09-11T17:59:07Z)
• Action-Constrained Imitation Learning [12.316546911223263]
  Policy learning under action constraints plays a central role in ensuring safe behaviors in various robot control and resource allocation applications.<n>In this paper, we study a new problem setting termed Action-Constrained Imitation Learning (ACIL), where an action-constrained imitator aims to learn from a demonstrative expert with larger action space.<n>We tackle this mismatch through textittrajectory alignment and propose DTWIL, which replaces the original expert demonstrations with a surrogate dataset that follows similar state trajectories while adhering to the action constraints.
  arXiv  Detail & Related papers  (2025-08-20T03:19:07Z)
• Reinforcement Learning for Decision-Level Interception Prioritization in Drone Swarm Defense [51.736723807086385]
  We present a case study demonstrating the practical advantages of reinforcement learning in addressing this challenge.<n>We introduce a high-fidelity simulation environment that captures realistic operational constraints.<n>Agent learns to coordinate multiple effectors for optimal interception prioritization.<n>We evaluate the learned policy against a handcrafted rule-based baseline across hundreds of simulated attack scenarios.
  arXiv  Detail & Related papers  (2025-08-01T13:55:39Z)
• Action Space Reduction Strategies for Reinforcement Learning in Autonomous Driving [0.0]
  Reinforcement Learning (RL) offers a promising framework for autonomous driving.<n>Large and high-dimensional action spaces often used to support fine-grained control can impede training efficiency and increase exploration costs.<n>We introduce and evaluate two novel structured action space modification strategies for RL in autonomous driving.
  arXiv  Detail & Related papers  (2025-07-07T17:58:08Z)
• Enhancing Aerial Combat Tactics through Hierarchical Multi-Agent Reinforcement Learning [38.15185397658309]
  This work presents a Hierarchical Multi-Agent Reinforcement Learning framework for analyzing simulated air combat scenarios.<n>The objective is to identify effective Courses of Action that lead to mission success within preset simulations.
  arXiv  Detail & Related papers  (2025-05-13T22:13:48Z)
• Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning [125.61772424068903]
  Vision-and-language navigation (VLN) asks an agent to follow a given language instruction to navigate through a real 3D environment. We present a model-agnostic training paradigm, called Progressive Perturbation-aware Contrastive Learning (PROPER) to enhance the generalization ability of existing VLN agents.
  arXiv  Detail & Related papers  (2024-03-09T02:34:13Z)
• Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows [58.762959061522736]
  offline reinforcement learning aims to train a policy on a pre-recorded and fixed dataset without any additional environment interactions. We build upon recent works on learning policies in latent action spaces and use a special form of Normalizing Flows for constructing a generative model. We evaluate our method on various locomotion and navigation tasks, demonstrating that our approach outperforms recently proposed algorithms.
  arXiv  Detail & Related papers  (2022-11-20T21:57:10Z)
• Learning Pneumatic Non-Prehensile Manipulation with a Mobile Blower [30.032847855193864]
  blowing controller must continually adapt to unexpected changes from its actions. We introduce a multi-frequency version of the spatial action maps framework. This allows for efficient learning of vision-based policies that effectively combine high-level planning and low-level closed-loop control.
  arXiv  Detail & Related papers  (2022-04-05T17:55:58Z)
• Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents [120.91291581594773]
  We present a formal formulation of a cooperative multi-agent reinforcement learning system with unexpected crashes. We propose a coach-assisted multi-agent reinforcement learning framework, which introduces a virtual coach agent to adjust the crash rate during training. To the best of our knowledge, this work is the first to study the unexpected crashes in the multi-agent system.
  arXiv  Detail & Related papers  (2022-03-16T08:22:45Z)
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation [99.2543521972137]
  ReLMoGen is a framework that combines a learned policy to predict subgoals and a motion generator to plan and execute the motion needed to reach these subgoals. Our method is benchmarked on a diverse set of seven robotics tasks in photo-realistic simulation environments. ReLMoGen shows outstanding transferability between different motion generators at test time, indicating a great potential to transfer to real robots.
  arXiv  Detail & Related papers  (2020-08-18T08:05:15Z)
• Whole-Body Control of a Mobile Manipulator using End-to-End Reinforcement Learning [31.150823782805283]
  We propose an end-to-end Reinforcement Learning (RL) approach to Whole-Body Control (WBC) We compared our learned controller against a state-of-the-art sampling-based method in simulation and achieved faster overall mission times.
  arXiv  Detail & Related papers  (2020-02-25T21:21:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.