Automated Cyber Defense with Generalizable Graph-based Reinforcement Learning Agents

• URL: http://arxiv.org/abs/2509.16151v1
• Date: Fri, 19 Sep 2025 16:57:27 GMT
• Title: Automated Cyber Defense with Generalizable Graph-based Reinforcement Learning Agents
• Authors: Isaiah J. King, Benjamin Bowman, H. Howie Huang,
• Abstract summary: Deep reinforcement learning is emerging as a viable strategy for automated cyber defense.<n>In this work, we frame ACD as a two-player context-based partially observable Markov decision problem.<n>We show that this approach outperforms the state-of-the-art by a wide margin.
• Score: 7.45063623129985
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Deep reinforcement learning (RL) is emerging as a viable strategy for automated cyber defense (ACD). The traditional RL approach represents networks as a list of computers in various states of safety or threat. Unfortunately, these models are forced to overfit to specific network topologies, rendering them ineffective when faced with even small environmental perturbations. In this work, we frame ACD as a two-player context-based partially observable Markov decision problem with observations represented as attributed graphs. This approach allows our agents to reason through the lens of relational inductive bias. Agents learn how to reason about hosts interacting with other system entities in a more general manner, and their actions are understood as edits to the graph representing the environment. By introducing this bias, we will show that our agents can better reason about the states of networks and zero-shot adapt to new ones. We show that this approach outperforms the state-of-the-art by a wide margin, and makes our agents capable of defending never-before-seen networks against a wide range of adversaries in a variety of complex, and multi-agent environments.

Related papers

• Towards a Generalisable Cyber Defence Agent for Real-World Computer Networks [0.0]
  This research introduces a novel set of Topological Extensions for Reinforcement Learning Agents (TERLA)<n>TERLA provides generalisability for the defence of networks with differing topology and size, without the need for retraining.<n>Generalisability has been demonstrated by showing that all TERLA agents have the same network-agnostic neural network architecture.
  arXiv  Detail & Related papers  (2025-11-12T08:38:34Z)
• Agentic Web: Weaving the Next Web with AI Agents [109.13815627467514]
  The emergence of AI agents powered by large language models (LLMs) marks a pivotal shift toward the Agentic Web.<n>In this paradigm, agents interact directly with one another to plan, coordinate, and execute complex tasks on behalf of users.<n>We present a structured framework for understanding and building the Agentic Web.
  arXiv  Detail & Related papers  (2025-07-28T17:58:12Z)
• Exploiting Edge Features for Transferable Adversarial Attacks in Distributed Machine Learning [54.26807397329468]
  This work explores a previously overlooked vulnerability in distributed deep learning systems.<n>An adversary who intercepts the intermediate features transmitted between them can still pose a serious threat.<n>We propose an exploitation strategy specifically designed for distributed settings.
  arXiv  Detail & Related papers  (2025-07-09T20:09:00Z)
• An Attentive Graph Agent for Topology-Adaptive Cyber Defence [1.0812794909131096]
  We develop a custom version of the Cyber Operations Research Gym (CybORG) environment, encoding network state as a directed graph.<n>We employ a Graph Attention Network (GAT) architecture to process node, edge, and global features, and adapt its output to be compatible with policy gradient methods in reinforcement learning.<n>We demonstrate that GAT defensive policies can be trained using our low-level directed graph observations, even when unexpected connections arise during simulation.
  arXiv  Detail & Related papers  (2025-01-24T18:22:37Z)
• Multi-Objective Reinforcement Learning for Automated Resilient Cyber Defence [0.0]
  Cyber-attacks pose a security threat to military command and control networks, Intelligence, Surveillance, and Reconnaissance (ISR) systems, and civilian critical national infrastructure.<n>The use of artificial intelligence and autonomous agents in these attacks increases the scale, range, and complexity of this threat and the subsequent disruption they cause.<n> Autonomous Cyber Defence (ACD) agents aim to mitigate this threat by responding at machine speed and at the scale required to address the problem.
  arXiv  Detail & Related papers  (2024-11-26T16:51:52Z)
• Entity-based Reinforcement Learning for Autonomous Cyber Defence [0.22499166814992438]
  Key challenge for autonomous cyber defence is ensuring a defensive agent's ability to generalise across diverse network topologies and configurations.<n>Standard approaches to deep reinforcement learning expect fixed-size observation and action spaces.<n>In autonomous cyber defence, this makes it hard to develop agents that generalise to environments with network topologies different from those trained on.
  arXiv  Detail & Related papers  (2024-10-23T08:04:12Z)
• Structural Generalization in Autonomous Cyber Incident Response with Message-Passing Neural Networks and Reinforcement Learning [0.0]
  Retraining agents for small network changes costs time and energy. We create variants of the original network with different numbers of hosts and agents are tested without additional training. Agents using the default vector state representation perform better, but need to be specially trained on each network variant.
  arXiv  Detail & Related papers  (2024-07-08T09:34:22Z)
• Dissecting Adversarial Robustness of Multimodal LM Agents [70.2077308846307]
  We manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena.<n>We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search.<n>We also use ARE to rigorously evaluate how the robustness changes as new components are added.
  arXiv  Detail & Related papers  (2024-06-18T17:32:48Z)
• Decentralized Adversarial Training over Graphs [44.03711922549992]
  The vulnerability of machine learning models to adversarial attacks has been attracting attention in recent years.<n>We develop a decentralized adversarial framework for multiagent systems.
  arXiv  Detail & Related papers  (2023-03-23T15:05:16Z)
• Graph Neural Networks for Decentralized Multi-Agent Perimeter Defense [111.9039128130633]
  We develop an imitation learning framework that learns a mapping from defenders' local perceptions and their communication graph to their actions. We run perimeter defense games in scenarios with different team sizes and configurations to demonstrate the performance of the learned network.
  arXiv  Detail & Related papers  (2023-01-23T19:35:59Z)
• Discriminator-Free Generative Adversarial Attack [87.71852388383242]
  Agenerative-based adversarial attacks can get rid of this limitation. ASymmetric Saliency-based Auto-Encoder (SSAE) generates the perturbations. The adversarial examples generated by SSAE not only make thewidely-used models collapse, but also achieves good visual quality.
  arXiv  Detail & Related papers  (2021-07-20T01:55:21Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.