Co-Evolving Complexity: An Adversarial Framework for Automatic MARL Curricula

• URL: http://arxiv.org/abs/2509.03771v3
• Date: Tue, 04 Nov 2025 06:38:29 GMT
• Title: Co-Evolving Complexity: An Adversarial Framework for Automatic MARL Curricula
• Authors: Brennen Hill,
• Abstract summary: General-purpose intelligent agents are intrinsically linked to the environments in which they are trained.<n>Hand-crafted environments are finite and often contain implicit biases, limiting the potential for agents to develop truly generalizable and robust skills.<n>We propose a paradigm for generating a boundless and adaptive curriculum of challenges by framing the environment generation process as an adversarial game.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The advancement of general-purpose intelligent agents is intrinsically linked to the environments in which they are trained. While scaling models and datasets has yielded remarkable capabilities, scaling the complexity, diversity, and interactivity of environments remains a crucial bottleneck. Hand-crafted environments are finite and often contain implicit biases, limiting the potential for agents to develop truly generalizable and robust skills. In this work, we propose a paradigm for generating a boundless and adaptive curriculum of challenges by framing the environment generation process as an adversarial game. We introduce a system where a team of cooperative multi-agent defenders learns to survive against a procedurally generative attacker. The attacker agent learns to produce increasingly challenging configurations of enemy units, dynamically creating novel worlds tailored to exploit the defenders' current weaknesses. Concurrently, the defender team learns cooperative strategies to overcome these generated threats. This co-evolutionary dynamic creates a self-scaling environment where complexity arises organically from the adversarial interaction, providing an effectively infinite stream of novel and relevant training data. We demonstrate that with minimal training, this approach leads to the emergence of complex, intelligent behaviors, such as flanking and shielding by the attacker, and focus-fire and spreading by the defenders. Our findings suggest that adversarial co-evolution is a powerful mechanism for automatically scaling environmental complexity, driving agents towards greater robustness and strategic depth.

Related papers

• Robust Agents in Open-Ended Worlds [4.199586801784625]
  In this thesis, we harness methodologies from open-endedness and multi-agent learning to train and evaluate robust AI agents.<n>We begin by introducing MiniHack, a sandbox framework for creating diverse environments through procedural content generation.<n>We then present Maestro, a novel approach for generating adversarial curricula that progressively enhance the robustness and generality of RL agents in two-player zero-sum games.
  arXiv  Detail & Related papers  (2025-12-09T00:30:33Z)
• Searching for Privacy Risks in LLM Agents via Simulation [61.229785851581504]
  We present a search-based framework that alternates between improving attack and defense strategies through the simulation of privacy-critical agent interactions.<n>We find that attack strategies escalate from direct requests to sophisticated tactics, such as impersonation and consent forgery.<n>The discovered attacks and defenses transfer across diverse scenarios and backbone models, demonstrating strong practical utility for building privacy-aware agents.
  arXiv  Detail & Related papers  (2025-08-14T17:49:09Z)
• A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence [87.08051686357206]
  Large Language Models (LLMs) have demonstrated strong capabilities but remain fundamentally static.<n>As LLMs are increasingly deployed in open-ended, interactive environments, this static nature has become a critical bottleneck.<n>This survey provides the first systematic and comprehensive review of self-evolving agents.
  arXiv  Detail & Related papers  (2025-07-28T17:59:05Z)
• GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspective [35.589506360952925]
  We argue for a transformative paradigm shift from reactive to proactive multi-agent intelligence through generative AI-based reinforcement learning.<n>Rather than responding to immediate observations, generative-RL agents can model environment evolution, predict other agents' behaviors, generate coordinated action sequences, and engage in strategic reasoning accounting for long-term dynamics.
  arXiv  Detail & Related papers  (2025-07-13T05:02:43Z)
• World Models for Cognitive Agents: Transforming Edge Intelligence in Future Networks [55.90051810762702]
  We present a comprehensive overview of world models, highlighting their architecture, training paradigms, and applications across prediction, generation, planning, and causal reasoning.<n>We propose Wireless Dreamer, a novel world model-based reinforcement learning framework tailored for wireless edge intelligence optimization.
  arXiv  Detail & Related papers  (2025-05-31T06:43:00Z)
• Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems [132.77459963706437]
  This book provides a comprehensive overview, framing intelligent agents within modular, brain-inspired architectures.<n>It explores self-enhancement and adaptive evolution mechanisms, exploring how agents autonomously refine their capabilities.<n>It also examines the collective intelligence emerging from agent interactions, cooperation, and societal structures.
  arXiv  Detail & Related papers  (2025-03-31T18:00:29Z)
• AdaWorld: Learning Adaptable World Models with Latent Actions [76.50869178593733]
  We propose AdaWorld, an innovative world model learning approach that enables efficient adaptation.<n>Key idea is to incorporate action information during the pretraining of world models.<n>We then develop an autoregressive world model that conditions on these latent actions.
  arXiv  Detail & Related papers  (2025-03-24T17:58:15Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer.<n>By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)
• Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI [52.138044013005]
  generative AI, particularly large language models (LLMs), become increasingly integrated into production applications. New attack surfaces and vulnerabilities emerge and put a focus on adversarial threats in natural language and multi-modal systems. Red-teaming has gained importance in proactively identifying weaknesses in these systems, while blue-teaming works to protect against such adversarial attacks. This work aims to bridge the gap between academic insights and practical security measures for the protection of generative AI systems.
  arXiv  Detail & Related papers  (2024-09-23T10:18:10Z)
• Understanding the World to Solve Social Dilemmas Using Multi-Agent Reinforcement Learning [0.7161783472741748]
  We study the behavior of self-interested rational agents that learn world models in a multi-agent reinforcement learning setting. Our simulation results show that groups of agents endowed with world models outperform all the other tested ones when dealing with scenarios where social dilemmas can arise.
  arXiv  Detail & Related papers  (2023-05-19T00:31:26Z)
• Emergent collective intelligence from massive-agent cooperation and competition [19.75488604218965]
  We study the emergence of artificial collective intelligence through massive-agent reinforcement learning. We propose a new massive-agent reinforcement learning environment, Lux, where dynamic and massive agents in two teams scramble for limited resources and fight off the darkness.
  arXiv  Detail & Related papers  (2023-01-04T13:23:12Z)
• Conditional Imitation Learning for Multi-Agent Games [89.897635970366]
  We study the problem of conditional multi-agent imitation learning, where we have access to joint trajectory demonstrations at training time. We propose a novel approach to address the difficulties of scalability and data scarcity. Our model learns a low-rank subspace over ego and partner agent strategies, then infers and adapts to a new partner strategy by interpolating in the subspace.
  arXiv  Detail & Related papers  (2022-01-05T04:40:13Z)
• Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design [121.73425076217471]
  We propose Unsupervised Environment Design (UED), where developers provide environments with unknown parameters, and these parameters are used to automatically produce a distribution over valid, solvable environments. We call our technique Protagonist Antagonist Induced Regret Environment Design (PAIRED) Our experiments demonstrate that PAIRED produces a natural curriculum of increasingly complex environments, and PAIRED agents achieve higher zero-shot transfer performance when tested in highly novel environments.
  arXiv  Detail & Related papers  (2020-12-03T17:37:01Z)
• Heterogeneous Multi-Agent Reinforcement Learning for Unknown Environment Mapping [0.0]
  We present an actor-critic algorithm that allows a team of heterogeneous agents to learn decentralized control policies for covering an unknown environment. This task is of interest to national security and emergency response organizations that would like to enhance situational awareness in hazardous areas by deploying teams of unmanned aerial vehicles.
  arXiv  Detail & Related papers  (2020-10-06T12:23:05Z)
• Natural Emergence of Heterogeneous Strategies in Artificially Intelligent Competitive Teams [0.0]
  We develop a competitive multi agent environment called FortAttack in which two teams compete against each other. We observe a natural emergence of heterogeneous behavior amongst homogeneous agents when such behavior can lead to the team's success. We propose ensemble training, in which we utilize the evolved opponent strategies to train a single policy for friendly agents.
  arXiv  Detail & Related papers  (2020-07-06T22:35:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.