Obstacle Avoidance for UAS in Continuous Action Space Using Deep
Reinforcement Learning
- URL: http://arxiv.org/abs/2111.07037v1
- Date: Sat, 13 Nov 2021 04:44:53 GMT
- Title: Obstacle Avoidance for UAS in Continuous Action Space Using Deep
Reinforcement Learning
- Authors: Jueming Hu, Xuxi Yang, Weichang Wang, Peng Wei, Lei Ying, Yongming Liu
- Abstract summary: Obstacle avoidance for small unmanned aircraft is vital for the safety of future urban air mobility.
We propose a deep reinforcement learning algorithm based on Proximal Policy Optimization (PPO) to guide autonomous UAS to their destinations.
Results show that the proposed model can provide accurate and robust guidance and resolve conflict with a success rate of over 99%.
- Score: 9.891207216312937
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Obstacle avoidance for small unmanned aircraft is vital for the safety of
future urban air mobility (UAM) and Unmanned Aircraft System (UAS) Traffic
Management (UTM). There are many techniques for real-time robust drone
guidance, but many of them solve in discretized airspace and control, which
would require an additional path smoothing step to provide flexible commands
for UAS. To provide a safe and efficient computational guidance of operations
for unmanned aircraft, we explore the use of a deep reinforcement learning
algorithm based on Proximal Policy Optimization (PPO) to guide autonomous UAS
to their destinations while avoiding obstacles through continuous control. The
proposed scenario state representation and reward function can map the
continuous state space to continuous control for both heading angle and speed.
To verify the performance of the proposed learning framework, we conducted
numerical experiments with static and moving obstacles. Uncertainties
associated with the environments and safety operation bounds are investigated
in detail. Results show that the proposed model can provide accurate and robust
guidance and resolve conflict with a success rate of over 99%.
Related papers
- Intercepting Unauthorized Aerial Robots in Controlled Airspace Using Reinforcement Learning [2.519319150166215]
The proliferation of unmanned aerial vehicles (UAVs) in controlled airspace presents significant risks.
This work addresses the need for robust, adaptive systems capable of managing such threats through the use of Reinforcement Learning (RL)
We present a novel approach utilizing RL to train fixed-wing UAV pursuer agents for intercepting dynamic evader targets.
arXiv Detail & Related papers (2024-07-09T14:45:47Z) - RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum.
We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
arXiv Detail & Related papers (2024-05-07T23:32:36Z) - SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Controllable Adversaries [94.84458417662407]
We introduce SAFE-SIM, a novel diffusion-based controllable closed-loop safety-critical simulation framework.
We develop a novel approach to simulate safety-critical scenarios through an adversarial term in the denoising process.
We validate our framework empirically using the NuScenes dataset, demonstrating improvements in both realism and controllability.
arXiv Detail & Related papers (2023-12-31T04:14:43Z) - A Safer Vision-based Autonomous Planning System for Quadrotor UAVs with
Dynamic Obstacle Trajectory Prediction and Its Application with LLMs [6.747468447244154]
This paper proposes a vision-based planning system that combines tracking and trajectory prediction of dynamic obstacles to achieve efficient and reliable autonomous flight.
We conduct experiments in both simulation and real-world environments, and the results indicate that our approach can successfully detect and avoid obstacles in dynamic environments in real-time.
arXiv Detail & Related papers (2023-11-21T08:09:00Z) - Safe multi-agent motion planning under uncertainty for drones using
filtered reinforcement learning [6.783774261623415]
We present a tractable motion planner that builds upon the strengths of reinforcement learning and constrained-control-based trajectory planning.
The proposed approach yields a safe, real-time implementable, multi-agent motion planner that is simpler to train than methods based solely on learning.
arXiv Detail & Related papers (2023-10-31T18:09:26Z) - Toward collision-free trajectory for autonomous and pilot-controlled
unmanned aerial vehicles [1.018017727755629]
This study makes greater use of electronic conspicuity (EC) information made available by PilotAware Ltd in developing an advanced collision management methodology.
The merits of the DACM methodology have been demonstrated through extensive simulations and real-world field tests in avoiding mid-air collisions.
arXiv Detail & Related papers (2023-09-18T18:24:31Z) - Evaluation of Safety Constraints in Autonomous Navigation with Deep
Reinforcement Learning [62.997667081978825]
We compare two learnable navigation policies: safe and unsafe.
The safe policy takes the constraints into the account, while the other does not.
We show that the safe policy is able to generate trajectories with more clearance (distance to the obstacles) and makes less collisions while training without sacrificing the overall performance.
arXiv Detail & Related papers (2023-07-27T01:04:57Z) - Reinforcement Learning-Based Air Traffic Deconfliction [7.782300855058585]
This work focuses on automating the horizontal separation of two aircraft and presents the obstacle avoidance problem as a 2D surrogate optimization task.
Using Reinforcement Learning (RL), we optimize the avoidance policy and model the dynamics, interactions, and decision-making.
The proposed system generates a quick and achievable avoidance trajectory that satisfies the safety requirements.
arXiv Detail & Related papers (2023-01-05T00:37:20Z) - Evaluating Model-free Reinforcement Learning toward Safety-critical
Tasks [70.76757529955577]
This paper revisits prior work in this scope from the perspective of state-wise safe RL.
We propose Unrolling Safety Layer (USL), a joint method that combines safety optimization and safety projection.
To facilitate further research in this area, we reproduce related algorithms in a unified pipeline and incorporate them into SafeRL-Kit.
arXiv Detail & Related papers (2022-12-12T06:30:17Z) - Differentiable Control Barrier Functions for Vision-based End-to-End
Autonomous Driving [100.57791628642624]
We introduce a safety guaranteed learning framework for vision-based end-to-end autonomous driving.
We design a learning system equipped with differentiable control barrier functions (dCBFs) that is trained end-to-end by gradient descent.
arXiv Detail & Related papers (2022-03-04T16:14:33Z) - Chance-Constrained Trajectory Optimization for Safe Exploration and
Learning of Nonlinear Systems [81.7983463275447]
Learning-based control algorithms require data collection with abundant supervision for training.
We present a new approach for optimal motion planning with safe exploration that integrates chance-constrained optimal control with dynamics learning and feedback control.
arXiv Detail & Related papers (2020-05-09T05:57:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.