An Attentive Graph Agent for Topology-Adaptive Cyber Defence

• URL: http://arxiv.org/abs/2501.14700v3
• Date: Tue, 18 Feb 2025 18:19:07 GMT
• Title: An Attentive Graph Agent for Topology-Adaptive Cyber Defence
• Authors: Ilya Orson Sandoval, Isaac Symes Thompson, Vasilios Mavroudis, Chris Hicks,
• Abstract summary: We develop a custom version of the Cyber Operations Research Gym (CybORG) environment, encoding network state as a directed graph. We employ a Graph Attention Network (GAT) architecture to process node, edge, and global features, and adapt its output to be compatible with policy gradient methods in reinforcement learning. We demonstrate that GAT defensive policies can be trained using our low-level directed graph observations, even when unexpected connections arise during simulation.
• Score: 1.0812794909131096
• License:
• Abstract: As cyber threats grow increasingly sophisticated, reinforcement learning (RL) is emerging as a promising technique to create intelligent and adaptive cyber defense systems. However, most existing autonomous defensive agents have overlooked the inherent graph structure of computer networks subject to cyber attacks, potentially missing critical information and constraining their adaptability. To overcome these limitations, we developed a custom version of the Cyber Operations Research Gym (CybORG) environment, encoding network state as a directed graph with realistic low-level features. We employ a Graph Attention Network (GAT) architecture to process node, edge, and global features, and adapt its output to be compatible with policy gradient methods in RL. Our GAT-based approach offers key advantages over flattened alternatives: policies that demonstrate resilience to certain types of unexpected dynamic network topology changes, reasonable generalisation to networks of varying sizes within the same structural distribution, and interpretable defensive actions grounded in tangible network properties. We demonstrate that GAT defensive policies can be trained using our low-level directed graph observations, even when unexpected connections arise during simulation. Evaluations across networks of different sizes, but consistent subnetwork structure, show our policies achieve comparable performance to policies trained specifically for each network configuration. Our study contributes to the development of robust cyber defence systems that can better adapt to real-world network security challenges.

Related papers

• Simulation of Multi-Stage Attack and Defense Mechanisms in Smart Grids [2.0766068042442174]
  We introduce a simulation environment that replicates the power grid's infrastructure and communication dynamics. The framework generates diverse, realistic attack data to train machine learning algorithms for detecting and mitigating cyber threats. It also provides a controlled, flexible platform to evaluate emerging security technologies, including advanced decision support systems.
  arXiv  Detail & Related papers  (2024-12-09T07:07:17Z)
• Entity-based Reinforcement Learning for Autonomous Cyber Defence [0.22499166814992438]
  Key challenge for autonomous cyber defence is ensuring a defensive agent's ability to generalise across diverse network topologies and configurations. Standard approaches to deep reinforcement learning expect fixed-size observation and action spaces. In autonomous cyber defence, this makes it hard to develop agents that generalise to environments with network topologies different from those trained on.
  arXiv  Detail & Related papers  (2024-10-23T08:04:12Z)
• Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defense [7.967738380932909]
  We propose a hierarchical Proximal Policy Optimization (PPO) architecture that decomposes the cyber defense task into specific sub-tasks like network investigation and host recovery. Our approach involves training sub-policies for each sub-task using PPO enhanced with domain expertise. These sub-policies are then leveraged by a master defense policy that coordinates their selection to solve complex network defense tasks.
  arXiv  Detail & Related papers  (2024-10-22T18:35:05Z)
• What Planning Problems Can A Relational Neural Network Solve? [91.53684831950612]
  We present a circuit complexity analysis for relational neural networks representing policies for planning problems. We show that there are three general classes of planning problems, in terms of the growth of circuit width and depth. We also illustrate the utility of this analysis for designing neural networks for policy learning.
  arXiv  Detail & Related papers  (2023-12-06T18:47:28Z)
• Everything Perturbed All at Once: Enabling Differentiable Graph Attacks [61.61327182050706]
  Graph neural networks (GNNs) have been shown to be vulnerable to adversarial attacks. We propose a novel attack method called Differentiable Graph Attack (DGA) to efficiently generate effective attacks. Compared to the state-of-the-art, DGA achieves nearly equivalent attack performance with 6 times less training time and 11 times smaller GPU memory footprint.
  arXiv  Detail & Related papers  (2023-08-29T20:14:42Z)
• Multi-agent Reinforcement Learning with Graph Q-Networks for Antenna Tuning [60.94661435297309]
  The scale of mobile networks makes it challenging to optimize antenna parameters using manual intervention or hand-engineered strategies. We propose a new multi-agent reinforcement learning algorithm to optimize mobile network configurations globally. We empirically demonstrate the performance of the algorithm on an antenna tilt tuning problem and a joint tilt and power control problem in a simulated environment.
  arXiv  Detail & Related papers  (2023-01-20T17:06:34Z)
• Dynamics-aware Adversarial Attack of Adaptive Neural Networks [75.50214601278455]
  We investigate the dynamics-aware adversarial attack problem of adaptive neural networks. We propose a Leaded Gradient Method (LGM) and show the significant effects of the lagged gradient. Our LGM achieves impressive adversarial attack performance compared with the dynamic-unaware attack methods.
  arXiv  Detail & Related papers  (2022-10-15T01:32:08Z)
• Dynamic Network Reconfiguration for Entropy Maximization using Deep Reinforcement Learning [3.012947865628207]
  Key problem in network theory is how to reconfigure a graph in order to optimize a quantifiable objective. In this paper, we cast the problem of network rewiring for optimizing a specified structural property as a Markov Decision Process (MDP) We then propose a general approach based on the Deep Q-Network (DQN) algorithm and graph neural networks (GNNs) that can efficiently learn strategies for rewiring networks.
  arXiv  Detail & Related papers  (2022-05-26T18:44:22Z)
• Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS [70.60975663021952]
  We study blackbox adversarial attacks on network classifiers. We argue that attacker-defender fixed points are themselves general-sum games with complex phase transitions. We show that a continual learning approach is required to study attacker-defender dynamics.
  arXiv  Detail & Related papers  (2021-11-23T23:42:16Z)
• The Feasibility and Inevitability of Stealth Attacks [63.14766152741211]
  We study new adversarial perturbations that enable an attacker to gain control over decisions in generic Artificial Intelligence systems. In contrast to adversarial data modification, the attack mechanism we consider here involves alterations to the AI system itself.
  arXiv  Detail & Related papers  (2021-06-26T10:50:07Z)
• Improving Neural Network Robustness through Neighborhood Preserving Layers [0.751016548830037]
  We demonstrate a novel neural network architecture which can incorporate such layers and also can be trained efficiently. We empirically show that our designed network architecture is more robust against state-of-art gradient descent based attacks.
  arXiv  Detail & Related papers  (2021-01-28T01:26:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.