Enhancing Aerial Combat Tactics through Hierarchical Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2505.08995v1
• Date: Tue, 13 May 2025 22:13:48 GMT
• Title: Enhancing Aerial Combat Tactics through Hierarchical Multi-Agent Reinforcement Learning
• Authors: Ardian Selmonaj, Oleg Szehr, Giacomo Del Rio, Alessandro Antonucci, Adrian Schneider, Michael Rüegsegger,
• Abstract summary: This work presents a Hierarchical Multi-Agent Reinforcement Learning framework for analyzing simulated air combat scenarios.<n>The objective is to identify effective Courses of Action that lead to mission success within preset simulations.
• Score: 38.15185397658309
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work presents a Hierarchical Multi-Agent Reinforcement Learning framework for analyzing simulated air combat scenarios involving heterogeneous agents. The objective is to identify effective Courses of Action that lead to mission success within preset simulations, thereby enabling the exploration of real-world defense scenarios at low cost and in a safe-to-fail setting. Applying deep Reinforcement Learning in this context poses specific challenges, such as complex flight dynamics, the exponential size of the state and action spaces in multi-agent systems, and the capability to integrate real-time control of individual units with look-ahead planning. To address these challenges, the decision-making process is split into two levels of abstraction: low-level policies control individual units, while a high-level commander policy issues macro commands aligned with the overall mission targets. This hierarchical structure facilitates the training process by exploiting policy symmetries of individual agents and by separating control from command tasks. The low-level policies are trained for individual combat control in a curriculum of increasing complexity. The high-level commander is then trained on mission targets given pre-trained control policies. The empirical validation confirms the advantages of the proposed framework.

Related papers

• Reinforcement Learning for Decision-Level Interception Prioritization in Drone Swarm Defense [56.47577824219207]
  We present a case study demonstrating the practical advantages of reinforcement learning in addressing this challenge.<n>We introduce a high-fidelity simulation environment that captures realistic operational constraints.<n>Agent learns to coordinate multiple effectors for optimal interception prioritization.<n>We evaluate the learned policy against a handcrafted rule-based baseline across hundreds of simulated attack scenarios.
  arXiv  Detail & Related papers  (2025-08-01T13:55:39Z)
• Decentralized Consensus Inference-based Hierarchical Reinforcement Learning for Multi-Constrained UAV Pursuit-Evasion Game [0.0]
  The Cooperative Evasion and Formation Coverage task belongs to one of the most challenging issues in pursuit-evasion games (MC-PEG)<n>We propose a novel two-level framework, which delegates localization to a high-level policy, while adopting a low-level policy to manage obstacle avoidance, navigation, and formation.<n>The experimental results, including the high-fidelity software-in-the-loop (SITL) simulations, validate that CI-HRL provides a superior solution with enhanced swarm's collaborative evasion and task completion capabilities.
  arXiv  Detail & Related papers  (2025-06-22T18:23:58Z)
• COMBO-Grasp: Learning Constraint-Based Manipulation for Bimanual Occluded Grasping [56.907940167333656]
  Occluded robot grasping is where the desired grasp poses are kinematically infeasible due to environmental constraints such as surface collisions.<n>Traditional robot manipulation approaches struggle with the complexity of non-prehensile or bimanual strategies commonly used by humans.<n>We introduce Constraint-based Manipulation for Bimanual Occluded Grasping (COMBO-Grasp), a learning-based approach which leverages two coordinated policies.
  arXiv  Detail & Related papers  (2025-02-12T01:31:01Z)
• A Hierarchical Reinforcement Learning Framework for Multi-UAV Combat Using Leader-Follower Strategy [3.095786524987445]
  Multi-UAV air combat is a complex task involving multiple autonomous UAVs.<n>Previous approaches predominantly discretize the action space into predefined actions.<n>We propose a hierarchical framework utilizing the Leader-Follower Multi-Agent Proximal Policy Optimization strategy.
  arXiv  Detail & Related papers  (2025-01-22T02:41:36Z)
• Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering [40.06500618820166]
  We propose a hierarchical multi-agent reinforcement learning framework for air-to-air combat with multiple heterogeneous agents. Low-level policies are trained for accurate unit combat control. The commander policy is trained on mission targets given pre-trained low-level policies.
  arXiv  Detail & Related papers  (2023-09-20T12:16:00Z)
• Learning Complex Teamwork Tasks Using a Given Sub-task Decomposition [11.998708550268978]
  We propose an approach which uses an expert-provided decomposition of a task into simpler multi-agent sub-tasks. In each sub-task, a subset of the entire team is trained to acquire sub-task-specific policies. The sub-teams are then merged and transferred to the target task, where their policies are collectively fine-tuned to solve the more complex target task.
  arXiv  Detail & Related papers  (2023-02-09T21:24:56Z)
• Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation [13.670618752160594]
  Deep reinforcement learning (DRL) provides a promising approach for multi-agent cooperation through the interaction of the agents and environments. Traditional DRL solutions suffer from the high dimensions of multiple agents with continuous action space during policy search. We propose a hierarchical reinforcement learning approach with high-level decision-making and low-level individual control for efficient policy search.
  arXiv  Detail & Related papers  (2022-06-25T19:09:29Z)
• Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space [76.46113138484947]
  General-purpose robots require diverse repertoires of behaviors to complete challenging tasks in real-world unstructured environments. To address this issue, goal-conditioned reinforcement learning aims to acquire policies that can reach goals for a wide range of tasks on command. We propose Planning to Practice, a method that makes it practical to train goal-conditioned policies for long-horizon tasks.
  arXiv  Detail & Related papers  (2022-05-17T06:58:17Z)
• Reinforcement Learning for Location-Aware Scheduling [1.0660480034605238]
  We show how various aspects of the warehouse environment affect performance and execution priority. We propose a compact representation of the state and action space for location-aware multi-agent systems. We also show how agents trained in certain environments maintain performance in completely unseen settings.
  arXiv  Detail & Related papers  (2022-03-07T15:51:00Z)
• RODE: Learning Roles to Decompose Multi-Agent Tasks [69.56458960841165]
  Role-based learning holds the promise of achieving scalable multi-agent learning by decomposing complex tasks using roles. We propose to first decompose joint action spaces into restricted role action spaces by clustering actions according to their effects on the environment and other agents. By virtue of these advances, our method outperforms the current state-of-the-art MARL algorithms on 10 of the 14 scenarios that comprise the challenging StarCraft II micromanagement benchmark.
  arXiv  Detail & Related papers  (2020-10-04T09:20:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.