PoolFlip: A Multi-Agent Reinforcement Learning Security Environment for Cyber Defense

• URL: http://arxiv.org/abs/2508.19488v1
• Date: Wed, 27 Aug 2025 00:18:49 GMT
• Title: PoolFlip: A Multi-Agent Reinforcement Learning Security Environment for Cyber Defense
• Abstract summary: The FlipIt game provides a framework for modeling interactions between a defender and an advanced adversary.<n>In FlipIt, the attacker and defender compete to control a shared resource by performing a foundational action and paying a cost.<n>We introduce PoolFlip, a multi-agent gym environment that extends the FlipIt game to allow efficient learning for attackers and defenders.
• Score: 8.817401748976375
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cyber defense requires automating defensive decision-making under stealthy, deceptive, and continuously evolving adversarial strategies. The FlipIt game provides a foundational framework for modeling interactions between a defender and an advanced adversary that compromises a system without being immediately detected. In FlipIt, the attacker and defender compete to control a shared resource by performing a Flip action and paying a cost. However, the existing FlipIt frameworks rely on a small number of heuristics or specialized learning techniques, which can lead to brittleness and the inability to adapt to new attacks. To address these limitations, we introduce PoolFlip, a multi-agent gym environment that extends the FlipIt game to allow efficient learning for attackers and defenders. Furthermore, we propose Flip-PSRO, a multi-agent reinforcement learning (MARL) approach that leverages population-based training to train defender agents equipped to generalize against a range of unknown, potentially adaptive opponents. Our empirical results suggest that Flip-PSRO defenders are $2\times$ more effective than baselines to generalize to a heuristic attack not exposed in training. In addition, our newly designed ownership-based utility functions ensure that Flip-PSRO defenders maintain a high level of control while optimizing performance.

Related papers

• Game-Theoretic Modeling of Stealthy Intrusion Defense against MDP-Based Attackers [1.230067722774416]
  We model advanced persistent threats as a strategic interaction between an attacker and a defender on an attack graph.<n>With limited information about the attacker's position and progress, the defender acts at random intervals by deploying intrusion detection sensors.<n>We derive optimal defensive strategies by solving the corresponding optimization problems.
  arXiv  Detail & Related papers  (2026-03-10T12:35:57Z)
• MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety [28.246225272659917]
  This paper introduces textbfMAGIC, a novel multi-turn multi-agent reinforcement learning framework.<n>It formulates Large Language Models safety alignment as an adversarial asymmetric game.<n>Our framework demonstrates superior defense success rates without compromising the helpfulness of the model.
  arXiv  Detail & Related papers  (2026-02-02T02:12:28Z)
• Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replay [19.431152130507648]
  Large Language Models (LLMs) have achieved remarkable capabilities but remain vulnerable to adversarial jailbreak'' attacks.<n>This paper introduces Safety Self- Play (SSP), a system that acts as both the Attacker (generating jailbreaks) and the Defender (refusing harmful requests)<n>SSP autonomously evolves robust defense capabilities, significantly outperforming baselines trained on static adversarial datasets.
  arXiv  Detail & Related papers  (2026-01-15T17:00:16Z)
• Safety Alignment of LMs via Non-cooperative Games [51.83432183158595]
  Current approaches rely on sequential adversarial training, generating adversarial prompts and fine-tuning LMs to defend against them.<n>We introduce a different paradigm: framing safety alignment as a non-zero-sum game between an Attacker LM and a Defender LM trained jointly.<n>Our method uses a preference-based reward signal derived from pairwise comparisons instead of point-wise scores, providing more robust supervision and potentially reducing reward hacking.
  arXiv  Detail & Related papers  (2025-12-23T22:13:14Z)
• Hammer and Anvil: A Principled Defense Against Backdoors in Federated Learning [19.849567299082306]
  Federated Learning is a distributed learning technique in which multiple clients cooperate to train a machine learning model.<n>In this work, we first devise a new adaptive adversary that surpasses existing adversaries in capabilities.<n>Then, we present Hammer and Anvil, a principled defense approach that combines two defenses in their underlying principle.
  arXiv  Detail & Related papers  (2025-09-09T18:54:31Z)
• Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models [55.28518567702213]
  Conventional language model (LM) safety alignment relies on a reactive, disjoint procedure: attackers exploit a static model, followed by defensive fine-tuning to patch exposed vulnerabilities.<n>This sequential approach creates a mismatch -- attackers overfit to obsolete defenses, while defenders perpetually lag behind emerging threats.<n>We propose Self-RedTeam, an online self-play reinforcement learning algorithm where an attacker and defender agent co-evolve through continuous interaction.
  arXiv  Detail & Related papers  (2025-06-09T06:35:12Z)
• Toward Optimal LLM Alignments Using Two-Player Games [86.39338084862324]
  In this paper, we investigate alignment through the lens of two-agent games, involving iterative interactions between an adversarial and a defensive agent. We theoretically demonstrate that this iterative reinforcement learning optimization converges to a Nash Equilibrium for the game induced by the agents. Experimental results in safety scenarios demonstrate that learning in such a competitive environment not only fully trains agents but also leads to policies with enhanced generalization capabilities for both adversarial and defensive agents.
  arXiv  Detail & Related papers  (2024-06-16T15:24:50Z)
• BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning [85.2564206440109]
  This paper reveals the threats in this practical scenario that backdoor attacks can remain effective even after defenses. We introduce the emphtoolns attack, which is resistant to backdoor detection and model fine-tuning defenses.
  arXiv  Detail & Related papers  (2023-11-20T02:21:49Z)
• Learning to Backdoor Federated Learning [9.046972927978997]
  In a federated learning (FL) system, malicious participants can easily embed backdoors into the aggregated model. We propose a general reinforcement learning-based backdoor attack framework. Our framework is both adaptive and flexible and achieves strong attack performance and durability even under state-of-the-art defenses.
  arXiv  Detail & Related papers  (2023-03-06T17:47:04Z)
• Fixed Points in Cyber Space: Rethinking Optimal Evasion Attacks in the Age of AI-NIDS [70.60975663021952]
  We study blackbox adversarial attacks on network classifiers. We argue that attacker-defender fixed points are themselves general-sum games with complex phase transitions. We show that a continual learning approach is required to study attacker-defender dynamics.
  arXiv  Detail & Related papers  (2021-11-23T23:42:16Z)
• Meta Federated Learning [57.52103907134841]
  Federated Learning (FL) is vulnerable to training time adversarial attacks. We propose Meta Federated Learning ( Meta-FL) which not only is compatible with secure aggregation protocol but also facilitates defense against backdoor attacks.
  arXiv  Detail & Related papers  (2021-02-10T16:48:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.