Reinforcement Learning-Based Air Traffic Deconfliction

• URL: http://arxiv.org/abs/2301.01861v1
• Date: Thu, 5 Jan 2023 00:37:20 GMT
• Title: Reinforcement Learning-Based Air Traffic Deconfliction
• Authors: Denis Osipychev, Dragos Margineantu, Girish Chowdhary
• Abstract summary: This work focuses on automating the horizontal separation of two aircraft and presents the obstacle avoidance problem as a 2D surrogate optimization task. Using Reinforcement Learning (RL), we optimize the avoidance policy and model the dynamics, interactions, and decision-making. The proposed system generates a quick and achievable avoidance trajectory that satisfies the safety requirements.
• Score: 7.782300855058585
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Remain Well Clear, keeping the aircraft away from hazards by the appropriate separation distance, is an essential technology for the safe operation of uncrewed aerial vehicles in congested airspace. This work focuses on automating the horizontal separation of two aircraft and presents the obstacle avoidance problem as a 2D surrogate optimization task. By our design, the surrogate task is made more conservative to guarantee the execution of the solution in the primary domain. Using Reinforcement Learning (RL), we optimize the avoidance policy and model the dynamics, interactions, and decision-making. By recursively sampling the resulting policy and the surrogate transitions, the system translates the avoidance policy into a complete avoidance trajectory. Then, the solver publishes the trajectory as a set of waypoints for the airplane to follow using the Robot Operating System (ROS) interface. The proposed system generates a quick and achievable avoidance trajectory that satisfies the safety requirements. Evaluation of our system is completed in a high-fidelity simulation and full-scale airplane demonstration. Moreover, the paper concludes an enormous integration effort that has enabled a real-life demonstration of the RL-based system.

Related papers

• Low-altitude Friendly-Jamming for Satellite-Maritime Communications via Generative AI-enabled Deep Reinforcement Learning [72.72954660774002]
  Low Earth Orbit (LEO) satellites can be used to assist maritime wireless communications for data transmission across wide-ranging areas. Extensive coverage of LEO satellites, combined with openness of channels, can cause the communication process to suffer from security risks. This paper presents a low-altitude friendly-jamming LEO satellite-maritime communication system enabled by a unmanned aerial vehicle.
  arXiv  Detail & Related papers  (2025-01-26T10:13:51Z)
• Task Delay and Energy Consumption Minimization for Low-altitude MEC via Evolutionary Multi-objective Deep Reinforcement Learning [52.64813150003228]
  The low-altitude economy (LAE), driven by unmanned aerial vehicles (UAVs) and other aircraft, has revolutionized fields such as transportation, agriculture, and environmental monitoring. In the upcoming six-generation (6G) era, UAV-assisted mobile edge computing (MEC) is particularly crucial in challenging environments such as mountainous or disaster-stricken areas. The task offloading problem is one of the key issues in UAV-assisted MEC, primarily addressing the trade-off between minimizing the task delay and the energy consumption of the UAV.
  arXiv  Detail & Related papers  (2025-01-11T02:32:42Z)
• Monocular Obstacle Avoidance Based on Inverse PPO for Fixed-wing UAVs [29.207513994002202]
  Fixed-wing Unmanned Aerial Vehicles (UAVs) are one of the most commonly used platforms for the Low-altitude Economy (LAE) and Urban Air Mobility (UAM) Classical obstacle avoidance systems, which rely on prior maps or sophisticated sensors, face limitations in unknown low-altitude environments and small UAV platforms. This paper proposes a lightweight deep reinforcement learning (DRL) based UAV collision avoidance system.
  arXiv  Detail & Related papers  (2024-11-27T03:03:37Z)
• ReGentS: Real-World Safety-Critical Driving Scenario Generation Made Stable [88.08120417169971]
  Machine learning based autonomous driving systems often face challenges with safety-critical scenarios that are rare in real-world data. This work explores generating safety-critical driving scenarios by modifying complex real-world regular scenarios through trajectory optimization. Our approach addresses unrealistic diverging trajectories and unavoidable collision scenarios that are not useful for training robust planner.
  arXiv  Detail & Related papers  (2024-09-12T08:26:33Z)
• SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
  We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework. Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations. We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
  arXiv  Detail & Related papers  (2023-12-31T04:14:43Z)
• Spacecraft Autonomous Decision-Planning for Collision Avoidance: a Reinforcement Learning Approach [0.0]
  This work proposes an implementation of autonomous CA decision-making capabilities on spacecraft based on Reinforcement Learning techniques. The proposed framework considers imperfect monitoring information about the status of the debris in orbit and allows the AI system to effectively learn policies to perform accurate Collision Avoidance Maneuvers (CAMs) The objective is to successfully delegate the decision-making process for autonomously implementing a CAM to the spacecraft without human intervention.
  arXiv  Detail & Related papers  (2023-10-29T10:15:33Z)
• In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution States [84.24300005271185]
  We propose a control filter that wraps any reference policy and effectively encourages the system to stay in-distribution with respect to offline-collected safe demonstrations. Our method is effective for two different visuomotor control tasks in simulation environments, including both top-down and egocentric view settings.
  arXiv  Detail & Related papers  (2023-01-27T22:28:19Z)
• Reward Function Optimization of a Deep Reinforcement Learning Collision Avoidance System [0.0]
  The proliferation of unmanned aircraft systems (UAS) has caused airspace regulation authorities to examine the interoperability of these aircraft with collision avoidance systems. Limitations in the currently mandated TCAS led the Federal Aviation Administration to commission the development of a new solution, the Airborne Collision Avoidance System X (ACAS X) This work explores the benefits of using a DRL collision avoidance system whose parameters are tuned using a surrogate.
  arXiv  Detail & Related papers  (2022-12-01T20:20:41Z)
• Smooth Trajectory Collision Avoidance through Deep Reinforcement Learning [0.0]
  We propose several novel agent state and reward function designs to tackle two critical issues in DRL-based navigation solutions. Our model relies on margin reward and smoothness constraints to ensure UAVs fly smoothly while greatly reducing the chance of collision.
  arXiv  Detail & Related papers  (2022-10-12T16:27:32Z)
• Obstacle Avoidance for UAS in Continuous Action Space Using Deep Reinforcement Learning [9.891207216312937]
  Obstacle avoidance for small unmanned aircraft is vital for the safety of future urban air mobility. We propose a deep reinforcement learning algorithm based on Proximal Policy Optimization (PPO) to guide autonomous UAS to their destinations. Results show that the proposed model can provide accurate and robust guidance and resolve conflict with a success rate of over 99%.
  arXiv  Detail & Related papers  (2021-11-13T04:44:53Z)
• SABER: Data-Driven Motion Planner for Autonomously Navigating Heterogeneous Robots [112.2491765424719]
  We present an end-to-end online motion planning framework that uses a data-driven approach to navigate a heterogeneous robot team towards a global goal. We use model predictive control (SMPC) to calculate control inputs that satisfy robot dynamics, and consider uncertainty during obstacle avoidance with chance constraints. recurrent neural networks are used to provide a quick estimate of future state uncertainty considered in the SMPC finite-time horizon solution. A Deep Q-learning agent is employed to serve as a high-level path planner, providing the SMPC with target positions that move the robots towards a desired global goal.
  arXiv  Detail & Related papers  (2021-08-03T02:56:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.