Maneuver Decision-Making For Autonomous Air Combat Through Curriculum Learning And Reinforcement Learning With Sparse Rewards

• URL: http://arxiv.org/abs/2302.05838v1
• Date: Sun, 12 Feb 2023 02:29:12 GMT
• Title: Maneuver Decision-Making For Autonomous Air Combat Through Curriculum Learning And Reinforcement Learning With Sparse Rewards
• Authors: Yu-Jie Wei, Hong-Peng Zhang, Chang-Qiang Huang
• Abstract summary: Three curricula of air combat maneuver decision-making are designed: angle curriculum, distance curriculum and hybrid curriculum. The training results show that angle curriculum can increase the speed and stability of training, and improve the performance of the agent. The maneuver decision results are consistent with the characteristics of missile.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Reinforcement learning is an effective way to solve the decision-making problems. It is a meaningful and valuable direction to investigate autonomous air combat maneuver decision-making method based on reinforcement learning. However, when using reinforcement learning to solve the decision-making problems with sparse rewards, such as air combat maneuver decision-making, it costs too much time for training and the performance of the trained agent may not be satisfactory. In order to solve these problems, the method based on curriculum learning is proposed. First, three curricula of air combat maneuver decision-making are designed: angle curriculum, distance curriculum and hybrid curriculum. These courses are used to train air combat agents respectively, and compared with the original method without any curriculum. The training results show that angle curriculum can increase the speed and stability of training, and improve the performance of the agent; distance curriculum can increase the speed and stability of agent training; hybrid curriculum has a negative impact on training, because it makes the agent get stuck at local optimum. The simulation results show that after training, the agent can handle the situations where targets come from different directions, and the maneuver decision results are consistent with the characteristics of missile.

Related papers

• An Imitative Reinforcement Learning Framework for Autonomous Dogfight [20.150691753213817]
  Unmanned Combat Aerial Vehicle (UCAV) dogfight plays a decisive role on the aerial battlefields. This paper proposes a novel imitative reinforcement learning framework, which efficiently leverages expert data while enabling autonomous exploration. The proposed framework can learn a successful dogfight policy of 'pursuit-lock-launch' for UCAVs.
  arXiv  Detail & Related papers  (2024-06-17T13:59:52Z)
• Efficient Mitigation of Bus Bunching through Setter-Based Curriculum Learning [0.47518865271427785]
  We propose a novel approach to curriculum learning that uses a Setter Model to automatically generate an action space, adversary strength, and bunching strength. Our method for automated curriculum learning involves a curriculum that is dynamically chosen and learned by an adversary network.
  arXiv  Detail & Related papers  (2024-05-23T18:26:55Z)
• Maneuver Decision-Making Through Automatic Curriculum Reinforcement Learning Without Handcrafted Reward functions [0.0]
  We propose an automatic curriculum reinforcement learning method, which enables agents to learn effective decisions in air combat from scratch. The range of initial states are used for distinguishing curricula of different difficulty levels. As sub-tasks change, agents gradually learn to complete a series of sub-tasks from easy to difficult.
  arXiv  Detail & Related papers  (2023-07-12T13:20:18Z)
• Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
  We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents. We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead. Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
  arXiv  Detail & Related papers  (2023-06-27T17:58:39Z)
• Reinforcement Learning Based Self-play and State Stacking Techniques for Noisy Air Combat Environment [1.7403133838762446]
  The complexity of air combat arises from aggressive close-range maneuvers and agile enemy behaviors. In this study, we developed an air combat simulation, which provides noisy observations to the agents. We present a state stacking method for noisy RL environments as a noise reduction technique.
  arXiv  Detail & Related papers  (2023-03-06T12:23:23Z)
• Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents [120.91291581594773]
  We present a formal formulation of a cooperative multi-agent reinforcement learning system with unexpected crashes. We propose a coach-assisted multi-agent reinforcement learning framework, which introduces a virtual coach agent to adjust the crash rate during training. To the best of our knowledge, this work is the first to study the unexpected crashes in the multi-agent system.
  arXiv  Detail & Related papers  (2022-03-16T08:22:45Z)
• Reinforcement Learning for Robust Missile Autopilot Design [0.0]
  This work is pioneer in proposing Reinforcement Learning as a framework for flight control. Under TRPO's methodology, the collected experience is augmented according to HER, stored in a replay buffer and sampled according to its significance. Results show that it is possible both to achieve the optimal performance and to improve the agent's robustness to uncertainties.
  arXiv  Detail & Related papers  (2020-11-26T09:30:04Z)
• Learning Agile Locomotion via Adversarial Training [59.03007947334165]
  In this paper, we present a multi-agent learning system, in which a quadruped robot (protagonist) learns to chase another robot (adversary) while the latter learns to escape. We find that this adversarial training process not only encourages agile behaviors but also effectively alleviates the laborious environment design effort. In contrast to prior works that used only one adversary, we find that training an ensemble of adversaries, each of which specializes in a different escaping strategy, is essential for the protagonist to master agility.
  arXiv  Detail & Related papers  (2020-08-03T01:20:37Z)
• Safe Reinforcement Learning via Curriculum Induction [94.67835258431202]
  In safety-critical applications, autonomous agents may need to learn in an environment where mistakes can be very costly. Existing safe reinforcement learning methods make an agent rely on priors that let it avoid dangerous situations. This paper presents an alternative approach inspired by human teaching, where an agent learns under the supervision of an automatic instructor.
  arXiv  Detail & Related papers  (2020-06-22T10:48:17Z)
• Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning [81.12201426668894]
  We develop efficient reinforcement learning methods that acquire diverse skills without any reward function, and then repurpose these skills for downstream tasks. We show that our proposed algorithm provides substantial improvement in learning efficiency, making reward-free real-world training feasible. We also demonstrate that the learned skills can be composed using model predictive control for goal-oriented navigation, without any additional training.
  arXiv  Detail & Related papers  (2020-04-27T17:38:53Z)
• Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads [69.21503033239985]
  Transporting suspended payloads is challenging for autonomous aerial vehicles. We propose a meta-learning approach that "learns how to learn" models of altered dynamics within seconds of post-connection flight data.
  arXiv  Detail & Related papers  (2020-04-23T17:43:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.