An Imitative Reinforcement Learning Framework for Autonomous Dogfight

• URL: http://arxiv.org/abs/2406.11562v1
• Date: Mon, 17 Jun 2024 13:59:52 GMT
• Title: An Imitative Reinforcement Learning Framework for Autonomous Dogfight
• Authors: Siyuan Li, Rongchang Zuo, Peng Liu, Yingnan Zhao,
• Abstract summary: Unmanned Combat Aerial Vehicle (UCAV) dogfight plays a decisive role on the aerial battlefields. This paper proposes a novel imitative reinforcement learning framework, which efficiently leverages expert data while enabling autonomous exploration. The proposed framework can learn a successful dogfight policy of 'pursuit-lock-launch' for UCAVs.
• Score: 20.150691753213817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unmanned Combat Aerial Vehicle (UCAV) dogfight, which refers to a fight between two or more UCAVs usually at close quarters, plays a decisive role on the aerial battlefields. With the evolution of artificial intelligence, dogfight progressively transits towards intelligent and autonomous modes. However, the development of autonomous dogfight policy learning is hindered by challenges such as weak exploration capabilities, low learning efficiency, and unrealistic simulated environments. To overcome these challenges, this paper proposes a novel imitative reinforcement learning framework, which efficiently leverages expert data while enabling autonomous exploration. The proposed framework not only enhances learning efficiency through expert imitation, but also ensures adaptability to dynamic environments via autonomous exploration with reinforcement learning. Therefore, the proposed framework can learn a successful dogfight policy of 'pursuit-lock-launch' for UCAVs. To support data-driven learning, we establish a dogfight environment based on the Harfang3D sandbox, where we conduct extensive experiments. The results indicate that the proposed framework excels in multistage dogfight, significantly outperforms state-of-the-art reinforcement learning and imitation learning methods. Thanks to the ability of imitating experts and autonomous exploration, our framework can quickly learn the critical knowledge in complex aerial combat tasks, achieving up to a 100% success rate and demonstrating excellent robustness.

Related papers

• Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning [50.33447711072726]
  This paper proposes a deep reinforcement learning-based model for decision-making in multi-role UAV cooperative pursuit-evasion game. The proposed method enables autonomous decision-making of the UAVs in pursuit-evasion game scenarios.
  arXiv  Detail & Related papers  (2024-11-05T10:45:30Z)
• Autonomous Agent for Beyond Visual Range Air Combat: A Deep Reinforcement Learning Approach [0.2578242050187029]
  This work contributes to developing an agent based on deep reinforcement learning capable of acting in a beyond visual range (BVR) air combat simulation environment. The paper presents an overview of building an agent representing a high-performance fighter aircraft that can learn and improve its role in BVR combat over time. It also hopes to examine a real pilot's ability, using virtual simulation, to interact in the same environment with the trained agent and compare their performances.
  arXiv  Detail & Related papers  (2023-04-19T13:54:37Z)
• Robust and Versatile Bipedal Jumping Control through Reinforcement Learning [141.56016556936865]
  This work aims to push the limits of agility for bipedal robots by enabling a torque-controlled bipedal robot to perform robust and versatile dynamic jumps in the real world. We present a reinforcement learning framework for training a robot to accomplish a large variety of jumping tasks, such as jumping to different locations and directions. We develop a new policy structure that encodes the robot's long-term input/output (I/O) history while also providing direct access to a short-term I/O history.
  arXiv  Detail & Related papers  (2023-02-19T01:06:09Z)
• Maneuver Decision-Making For Autonomous Air Combat Through Curriculum Learning And Reinforcement Learning With Sparse Rewards [0.0]
  Three curricula of air combat maneuver decision-making are designed: angle curriculum, distance curriculum and hybrid curriculum. The training results show that angle curriculum can increase the speed and stability of training, and improve the performance of the agent. The maneuver decision results are consistent with the characteristics of missile.
  arXiv  Detail & Related papers  (2023-02-12T02:29:12Z)
• DIAMBRA Arena: a New Reinforcement Learning Platform for Research and Experimentation [91.3755431537592]
  This work presents DIAMBRA Arena, a new platform for reinforcement learning research and experimentation. It features a collection of high-quality environments exposing a Python API fully compliant with OpenAI Gym standard. They are episodic tasks with discrete actions and observations composed by raw pixels plus additional numerical values.
  arXiv  Detail & Related papers  (2022-10-19T14:39:10Z)
• Harfang3D Dog-Fight Sandbox: A Reinforcement Learning Research Platform for the Customized Control Tasks of Fighter Aircrafts [0.0]
  We present a semi-realistic flight simulation environment Harfang3D Dog-Fight Sandbox for fighter aircrafts. It is aimed to be a flexible toolbox for the investigation of main challenges in aviation studies using Reinforcement Learning. Software also allows deployment of bot aircrafts and development of multi-agent tasks.
  arXiv  Detail & Related papers  (2022-10-13T18:18:09Z)
• A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Platform [0.0]
  We proposed a reinforcement learning framework based on Gazebo that is a kind of physical simulation platform (ROS-RL) We used three continuous action space reinforcement learning algorithms in the framework to dealing with the problem of autonomous landing of drones.
  arXiv  Detail & Related papers  (2022-09-07T06:33:57Z)
• Reinforcement Learning based Air Combat Maneuver Generation [0.0]
  In this research we aimed our UAV which has a dubin vehicle dynamic property to move to the target in two dimensional space in an optimal path. We did tests on two different environments and used simulations.
  arXiv  Detail & Related papers  (2022-01-14T15:55:44Z)
• Autonomous Reinforcement Learning: Formalism and Benchmarking [106.25788536376007]
  Real-world embodied learning, such as that performed by humans and animals, is situated in a continual, non-episodic world. Common benchmark tasks in RL are episodic, with the environment resetting between trials to provide the agent with multiple attempts. This discrepancy presents a major challenge when attempting to take RL algorithms developed for episodic simulated environments and run them on real-world platforms.
  arXiv  Detail & Related papers  (2021-12-17T16:28:06Z)
• Learning Agile Locomotion via Adversarial Training [59.03007947334165]
  In this paper, we present a multi-agent learning system, in which a quadruped robot (protagonist) learns to chase another robot (adversary) while the latter learns to escape. We find that this adversarial training process not only encourages agile behaviors but also effectively alleviates the laborious environment design effort. In contrast to prior works that used only one adversary, we find that training an ensemble of adversaries, each of which specializes in a different escaping strategy, is essential for the protagonist to master agility.
  arXiv  Detail & Related papers  (2020-08-03T01:20:37Z)
• Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning [81.12201426668894]
  We develop efficient reinforcement learning methods that acquire diverse skills without any reward function, and then repurpose these skills for downstream tasks. We show that our proposed algorithm provides substantial improvement in learning efficiency, making reward-free real-world training feasible. We also demonstrate that the learned skills can be composed using model predictive control for goal-oriented navigation, without any additional training.
  arXiv  Detail & Related papers  (2020-04-27T17:38:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.