Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defense

• URL: http://arxiv.org/abs/2410.17351v2
• Date: Thu, 24 Oct 2024 15:57:45 GMT
• Title: Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defense
• Authors: Aditya Vikram Singh, Ethan Rathbun, Emma Graham, Lisa Oakley, Simona Boboila, Alina Oprea, Peter Chin,
• Abstract summary: We propose a hierarchical Proximal Policy Optimization (PPO) architecture that decomposes the cyber defense task into specific sub-tasks like network investigation and host recovery. Our approach involves training sub-policies for each sub-task using PPO enhanced with domain expertise. These sub-policies are then leveraged by a master defense policy that coordinates their selection to solve complex network defense tasks.
• Score: 7.967738380932909
• License:
• Abstract: Recent advances in multi-agent reinforcement learning (MARL) have created opportunities to solve complex real-world tasks. Cybersecurity is a notable application area, where defending networks against sophisticated adversaries remains a challenging task typically performed by teams of security operators. In this work, we explore novel MARL strategies for building autonomous cyber network defenses that address challenges such as large policy spaces, partial observability, and stealthy, deceptive adversarial strategies. To facilitate efficient and generalized learning, we propose a hierarchical Proximal Policy Optimization (PPO) architecture that decomposes the cyber defense task into specific sub-tasks like network investigation and host recovery. Our approach involves training sub-policies for each sub-task using PPO enhanced with domain expertise. These sub-policies are then leveraged by a master defense policy that coordinates their selection to solve complex network defense tasks. Furthermore, the sub-policies can be fine-tuned and transferred with minimal cost to defend against shifts in adversarial behavior or changes in network settings. We conduct extensive experiments using CybORG Cage 4, the state-of-the-art MARL environment for cyber defense. Comparisons with multiple baselines across different adversaries show that our hierarchical learning approach achieves top performance in terms of convergence speed, episodic return, and several interpretable metrics relevant to cybersecurity, including the fraction of clean machines on the network, precision, and false positives on recoveries.

Related papers

• Multi-Objective Reinforcement Learning for Automated Resilient Cyber Defence [0.0]
  Cyber-attacks pose a security threat to military command and control networks, Intelligence, Surveillance, and Reconnaissance (ISR) systems, and civilian critical national infrastructure. The use of artificial intelligence and autonomous agents in these attacks increases the scale, range, and complexity of this threat and the subsequent disruption they cause. Autonomous Cyber Defence (ACD) agents aim to mitigate this threat by responding at machine speed and at the scale required to address the problem.
  arXiv  Detail & Related papers  (2024-11-26T16:51:52Z)
• Entity-based Reinforcement Learning for Autonomous Cyber Defence [0.22499166814992438]
  Key challenge for autonomous cyber defence is ensuring a defensive agent's ability to generalise across diverse network topologies and configurations. Standard approaches to deep reinforcement learning expect fixed-size observation and action spaces. In autonomous cyber defence, this makes it hard to develop agents that generalise to environments with network topologies different from those trained on.
  arXiv  Detail & Related papers  (2024-10-23T08:04:12Z)
• Mutual-modality Adversarial Attack with Semantic Perturbation [81.66172089175346]
  We propose a novel approach that generates adversarial attacks in a mutual-modality optimization scheme. Our approach outperforms state-of-the-art attack methods and can be readily deployed as a plug-and-play solution.
  arXiv  Detail & Related papers  (2023-12-20T05:06:01Z)
• On Autonomous Agents in a Cyber Defence Environment [0.0]
  We explore the utility of the autonomous cyber operation environments presented as part of the Cyber Autonomy Gym for Experimentation. CAGE Challenge 2 required a defensive Blue agent to defend a network from an attacking Red agent. We identify four classes of algorithms, namely, Single- Agent Deep Reinforcement Learning (DRL), Hierarchical DRL, Ensembles, and Non-DRL approaches.
  arXiv  Detail & Related papers  (2023-09-14T02:09:36Z)
• Baseline Defenses for Adversarial Attacks Against Aligned Language Models [109.75753454188705]
  Recent work shows that text moderations can produce jailbreaking prompts that bypass defenses. We look at three types of defenses: detection (perplexity based), input preprocessing (paraphrase and retokenization), and adversarial training. We find that the weakness of existing discretes for text, combined with the relatively high costs of optimization, makes standard adaptive attacks more challenging for LLMs.
  arXiv  Detail & Related papers  (2023-09-01T17:59:44Z)
• Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learning [4.796742432333795]
  Team of intelligent agents in computer network defence roles may reveal promising avenues to safeguard cyber and kinetic assets. Agents are evaluated on their ability to jointly mitigate attacker activity in host-based defence scenarios.
  arXiv  Detail & Related papers  (2023-08-25T14:07:50Z)
• Graph Neural Networks for Decentralized Multi-Agent Perimeter Defense [111.9039128130633]
  We develop an imitation learning framework that learns a mapping from defenders' local perceptions and their communication graph to their actions. We run perimeter defense games in scenarios with different team sizes and configurations to demonstrate the performance of the learned network.
  arXiv  Detail & Related papers  (2023-01-23T19:35:59Z)
• Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS [70.60975663021952]
  We study blackbox adversarial attacks on network classifiers. We argue that attacker-defender fixed points are themselves general-sum games with complex phase transitions. We show that a continual learning approach is required to study attacker-defender dynamics.
  arXiv  Detail & Related papers  (2021-11-23T23:42:16Z)
• Network Defense is Not a Game [0.0]
  Research seeks to apply Artificial Intelligence to scale and extend the capabilities of human operators to defend networks. Our position is that network defense is better characterized as a collection of games with uncertain and possibly drifting rules. We propose to define network defense tasks as distributions of network environments.
  arXiv  Detail & Related papers  (2021-04-20T21:52:51Z)
• Adversarial Machine Learning Attacks and Defense Methods in the Cyber Security Domain [58.30296637276011]
  This paper summarizes the latest research on adversarial attacks against security solutions based on machine learning techniques. It is the first to discuss the unique challenges of implementing end-to-end adversarial attacks in the cyber security domain.
  arXiv  Detail & Related papers  (2020-07-05T18:22:40Z)
• Dynamic Divide-and-Conquer Adversarial Training for Robust Semantic Segmentation [79.42338812621874]
  Adversarial training is promising for improving robustness of deep neural networks towards adversarial perturbations. We formulate a general adversarial training procedure that can perform decently on both adversarial and clean samples. We propose a dynamic divide-and-conquer adversarial training (DDC-AT) strategy to enhance the defense effect.
  arXiv  Detail & Related papers  (2020-03-14T05:06:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.