COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Game

• URL: http://arxiv.org/abs/2603.28386v1
• Date: Mon, 30 Mar 2026 12:56:54 GMT
• Title: COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Game
• Abstract summary: COvolve is a co-evolutionary framework for building continually improving agents.<n>It generates both environments and agent policies, expressed as executable Python code.<n>Our results demonstrate the potential of LLM-driven co-evolution to achieve open-ended learning.
• Score: 15.550671286269209
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A central challenge in building continually improving agents is that training environments are typically static or manually constructed. This restricts continual learning and generalization beyond the training distribution. We address this with COvolve, a co-evolutionary framework that leverages large language models (LLMs) to generate both environments and agent policies, expressed as executable Python code. We model the interaction between environment and policy designers as a two-player zero-sum game, ensuring adversarial co-evolution in which environments expose policy weaknesses and policies adapt in response. This process induces an automated curriculum in which environments and policies co-evolve toward increasing complexity. To guarantee robustness and prevent forgetting as the curriculum progresses, we compute the mixed-strategy Nash equilibrium (MSNE) of the zero-sum game, thereby yielding a meta-policy. This MSNE meta-policy ensures that the agent does not forget to solve previously seen environments while learning to solve previously unseen ones. Experiments in urban driving, symbolic maze-solving, and geometric navigation showcase that COvolve produces progressively more complex environments. Our results demonstrate the potential of LLM-driven co-evolution to achieve open-ended learning without predefined task distributions or manual intervention.

Related papers

• Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning [51.17262419591891]
  Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains.<n>We present MetaEvolve, a framework designed to develop these meta-skills via a data synthesis pipeline, evolution-aware reinforcement learning, and inference-time evolutionary search.
  arXiv  Detail & Related papers  (2026-07-24T04:35:29Z)
• Self-evolving LLM agents with in-distribution Optimization [60.05867547965365]
  Large Language Models (LLMs) have recently emerged as powerful controllers for interactive agents in complex environments.<n>We propose Q-Evolve, a self-evolving framework for LLM agents that unifies automatic process-reward labeling and policy learning.<n>We evaluate our method on AlfWorld, WebShop, and ScienceWorld, showing Q-Evolve outperforms strong baselines in sample efficiency, robustness, and overall task performance.
  arXiv  Detail & Related papers  (2026-06-05T15:09:52Z)
• Autonomous Continual Learning of Computer-Use Agents for Environment Adaptation [57.65688895630163]
  We introduce ACuRL, an Autonomous Curriculum Reinforcement Learning framework that continually adapts agents to specific environments with zero human data.<n>Our method effectively enables both intra-environment and cross-environment continual learning, yielding 4-22% performance gains without forgetting existing environments.
  arXiv  Detail & Related papers  (2026-02-10T23:06:02Z)
• Policy-Conditioned Policies for Multi-Agent Task Solving [53.67744322553693]
  In this work, we propose a paradigm shift that bridges the gap by representing policies as human-interpretable source code.<n>We reformulate the learning problem by utilizing Large Language Models (LLMs) as approximate interpreters.<n>We formalize this process as textitProgrammatic Iterated Best Response (PIBR), an algorithm where the policy code is optimized by textual gradients.
  arXiv  Detail & Related papers  (2025-12-24T07:42:10Z)
• Co-Evolving Complexity: An Adversarial Framework for Automatic MARL Curricula [0.0]
  General-purpose intelligent agents are intrinsically linked to the environments in which they are trained.<n>Hand-crafted environments are finite and often contain implicit biases, limiting the potential for agents to develop truly generalizable and robust skills.<n>We propose a paradigm for generating a boundless and adaptive curriculum of challenges by framing the environment generation process as an adversarial game.
  arXiv  Detail & Related papers  (2025-09-03T23:32:39Z)
• COMBO-Grasp: Learning Constraint-Based Manipulation for Bimanual Occluded Grasping [56.907940167333656]
  Occluded robot grasping is where the desired grasp poses are kinematically infeasible due to environmental constraints such as surface collisions.<n>Traditional robot manipulation approaches struggle with the complexity of non-prehensile or bimanual strategies commonly used by humans.<n>We introduce Constraint-based Manipulation for Bimanual Occluded Grasping (COMBO-Grasp), a learning-based approach which leverages two coordinated policies.
  arXiv  Detail & Related papers  (2025-02-12T01:31:01Z)
• Evolution of Societies via Reinforcement Learning [10.689403855269704]
  We propose a methodology that enables simulating populations of Reinforcement Learning agents at evolutionary scale.<n>We derive a fast, parallelizable implementation of Policy Gradient (PG) and Opponent-Learning Awareness (LOLA)<n>We demonstrate our approach by simulating the evolution of very large populations made of heterogeneous co-learning agents, under both naive and advanced learning strategies.
  arXiv  Detail & Related papers  (2024-10-22T22:49:04Z)
• Octopus: Embodied Vision-Language Programmer from Environmental Feedback [58.04529328728999]
  Embodied vision-language models (VLMs) have achieved substantial progress in multimodal perception and reasoning. To bridge this gap, we introduce Octopus, an embodied vision-language programmer that uses executable code generation as a medium to connect planning and manipulation. Octopus is designed to 1) proficiently comprehend an agent's visual and textual task objectives, 2) formulate intricate action sequences, and 3) generate executable code.
  arXiv  Detail & Related papers  (2023-10-12T17:59:58Z)
• MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learning [22.28076947612619]
  We introduce Multi-Agent Environment Design Strategist for Open-Ended Learning (MAESTRO) MAESTRO is the first multi-agent UED approach for two-player zero-sum settings. Our experiments show that MAESTRO outperforms a number of strong baselines on competitive two-player games.
  arXiv  Detail & Related papers  (2023-03-06T18:57:41Z)
• Dichotomy of Control: Separating What You Can Control from What You Cannot [129.62135987416164]
  We propose a future-conditioned supervised learning framework that separates mechanisms within a policy's control (actions) from those beyond a policy's control (environmentity) We show that DoC yields policies that are consistent with their conditioning inputs, ensuring that conditioning a learned policy on a desired high-return future outcome will correctly induce high-return behavior.
  arXiv  Detail & Related papers  (2022-10-24T17:49:56Z)
• Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization [29.61829620717385]
  Key challenge of continual reinforcement learning (CRL) in dynamic environments is to promptly adapt the RL agent's behavior as the environment changes over its lifetime. DaCoRL learns a context-conditioned policy using progressive contextualization. DaCoRL features consistent superiority over existing methods in terms of the stability, overall performance and generalization ability.
  arXiv  Detail & Related papers  (2022-09-01T10:26:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.