Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering

• URL: http://arxiv.org/abs/2309.11247v1
• Date: Wed, 20 Sep 2023 12:16:00 GMT
• Title: Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering
• Authors: Ardian Selmonaj, Oleg Szehr, Giacomo Del Rio, Alessandro Antonucci, Adrian Schneider, Michael R\"uegsegger
• Abstract summary: We propose a hierarchical multi-agent reinforcement learning framework for air-to-air combat with multiple heterogeneous agents. Low-level policies are trained for accurate unit combat control. The commander policy is trained on mission targets given pre-trained low-level policies.
• Score: 40.06500618820166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The application of artificial intelligence to simulate air-to-air combat scenarios is attracting increasing attention. To date the high-dimensional state and action spaces, the high complexity of situation information (such as imperfect and filtered information, stochasticity, incomplete knowledge about mission targets) and the nonlinear flight dynamics pose significant challenges for accurate air combat decision-making. These challenges are exacerbated when multiple heterogeneous agents are involved. We propose a hierarchical multi-agent reinforcement learning framework for air-to-air combat with multiple heterogeneous agents. In our framework, the decision-making process is divided into two stages of abstraction, where heterogeneous low-level policies control the action of individual units, and a high-level commander policy issues macro commands given the overall mission targets. Low-level policies are trained for accurate unit combat control. Their training is organized in a learning curriculum with increasingly complex training scenarios and league-based self-play. The commander policy is trained on mission targets given pre-trained low-level policies. The empirical validation advocates the advantages of our design choices.

Related papers

• Multi-granular Adversarial Attacks against Black-box Neural Ranking Models [111.58315434849047]
  We create high-quality adversarial examples by incorporating multi-granular perturbations. We transform the multi-granular attack into a sequential decision-making process. Our attack method surpasses prevailing baselines in both attack effectiveness and imperceptibility.
  arXiv  Detail & Related papers  (2024-04-02T02:08:29Z)
• A Dual Curriculum Learning Framework for Multi-UAV Pursuit-Evasion in Diverse Environments [15.959963737956848]
  This paper addresses multi-UAV pursuit-evasion, where a group of drones cooperate to capture a fast evader in a confined environment with obstacles. Existing algorithms, which simplify the pursuit-evasion problem, often lack expressive coordination strategies and struggle to capture the evader in extreme scenarios. We introduce a dual curriculum learning framework, named DualCL, which addresses multi-UAV pursuit-evasion in diverse environments and demonstrates zero-shot transfer ability to unseen scenarios.
  arXiv  Detail & Related papers  (2023-12-19T15:39:09Z)
• MaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learning [56.00558959816801]
  We propose a Mask-Based collaborative learning framework for Multi-Agent decision making (MaskMA) We show MaskMA can achieve an impressive 77.8% average zero-shot win rate on 60 unseen test maps by decentralized execution.
  arXiv  Detail & Related papers  (2023-10-18T09:53:27Z)
• Maneuver Decision-Making Through Automatic Curriculum Reinforcement Learning Without Handcrafted Reward functions [0.0]
  We propose an automatic curriculum reinforcement learning method, which enables agents to learn effective decisions in air combat from scratch. The range of initial states are used for distinguishing curricula of different difficulty levels. As sub-tasks change, agents gradually learn to complete a series of sub-tasks from easy to difficult.
  arXiv  Detail & Related papers  (2023-07-12T13:20:18Z)
• Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers [23.15190337027283]
  We propose Robust Multi-Agent Coordination via Generation of Auxiliary Adversarial Attackers (ROMANCE) ROMANCE enables the trained policy to encounter diversified and strong auxiliary adversarial attacks during training, thus achieving high robustness under various policy perturbations. The goal of quality is to minimize the ego-system coordination effect, and a novel diversity regularizer is applied to diversify the behaviors among attackers.
  arXiv  Detail & Related papers  (2023-05-10T05:29:47Z)
• Robust and Versatile Bipedal Jumping Control through Reinforcement Learning [141.56016556936865]
  This work aims to push the limits of agility for bipedal robots by enabling a torque-controlled bipedal robot to perform robust and versatile dynamic jumps in the real world. We present a reinforcement learning framework for training a robot to accomplish a large variety of jumping tasks, such as jumping to different locations and directions. We develop a new policy structure that encodes the robot's long-term input/output (I/O) history while also providing direct access to a short-term I/O history.
  arXiv  Detail & Related papers  (2023-02-19T01:06:09Z)
• Learning to Guide Multiple Heterogeneous Actors from a Single Human Demonstration via Automatic Curriculum Learning in StarCraft II [0.5911087507716211]
  In this work, we aim to train deep reinforcement learning agents that can command multiple heterogeneous actors. Our results show that an agent trained via automated curriculum learning can outperform state-of-the-art deep reinforcement learning baselines.
  arXiv  Detail & Related papers  (2022-05-11T21:53:11Z)
• Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS [70.60975663021952]
  We study blackbox adversarial attacks on network classifiers. We argue that attacker-defender fixed points are themselves general-sum games with complex phase transitions. We show that a continual learning approach is required to study attacker-defender dynamics.
  arXiv  Detail & Related papers  (2021-11-23T23:42:16Z)
• Room Clearance with Feudal Hierarchical Reinforcement Learning [2.867517731896504]
  We introduce a new simulation environment, "it", designed as a tool to build scenarios that can drive RL research in a direction useful for military analysis. We focus on an abstracted and simplified room clearance scenario, where a team of blue agents have to make their way through a building and ensure that all rooms are cleared of enemy red agents. We implement a multi-agent version of feudal hierarchical RL that introduces a command hierarchy where a commander at the higher level sends orders to multiple agents at the lower level who simply have to learn to follow these orders. We find that breaking the task down in this way allows us to
  arXiv  Detail & Related papers  (2021-05-24T15:05:58Z)
• RODE: Learning Roles to Decompose Multi-Agent Tasks [69.56458960841165]
  Role-based learning holds the promise of achieving scalable multi-agent learning by decomposing complex tasks using roles. We propose to first decompose joint action spaces into restricted role action spaces by clustering actions according to their effects on the environment and other agents. By virtue of these advances, our method outperforms the current state-of-the-art MARL algorithms on 10 of the 14 scenarios that comprise the challenging StarCraft II micromanagement benchmark.
  arXiv  Detail & Related papers  (2020-10-04T09:20:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.