Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Tasks

• URL: http://arxiv.org/abs/2604.20987v1
• Date: Wed, 22 Apr 2026 18:17:17 GMT
• Title: Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Tasks
• Abstract summary: Large Language Models (LLMs) offer a promising alternative as game playing agents, but they often struggle with consistent long horizon decision making.<n>We present COSPLAY, a co evolution framework in which an LLM decision agent retrieves skills from a learnable skill bank to guide action taking.<n>Our framework improves both the decision agent to learn better skill retrieval and action generation, while the skill bank agent continually extracts, refines, and updates skills together with their contracts.
• Score: 83.4942519570046
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long horizon interactive environments are a testbed for evaluating agents skill usage abilities. These environments demand multi step reasoning, the chaining of multiple skills over many timesteps, and robust decision making under delayed rewards and partial observability. Games are a good testbed for evaluating agent skill usage in environments. Large Language Models (LLMs) offer a promising alternative as game playing agents, but they often struggle with consistent long horizon decision making because they lack a mechanism to discover, retain, and reuse structured skills across episodes. We present COSPLAY, a co evolution framework in which an LLM decision agent retrieves skills from a learnable skill bank to guide action taking, while an agent managed skill pipeline discovers reusable skills from the agents unlabeled rollouts to form a skill bank. Our framework improves both the decision agent to learn better skill retrieval and action generation, while the skill bank agent continually extracts, refines, and updates skills together with their contracts. Experiments across six game environments show that COSPLAY with an 8B base model achieves over 25.1 percent average reward improvement against four frontier LLM baselines on single player game benchmarks while remaining competitive on multi player social reasoning games.

Related papers

• MASkills: Continual Skills Optimization for Multi-Agent LLM Systems [36.30615078891684]
  We introduce MASkills, a continual learning framework that optimize multi-agent LLM systems through agent skills.<n>We present a new agent-optimization pipeline that integrates skill-conditioned credit assignment, hierarchical credit aggregation, and momentum-smoothed optimization.<n>Experiments on HotpotQA, LoCoMo, and GAIA demonstrate the effectiveness of MASkills across multiple agentic tasks.
  arXiv  Detail & Related papers  (2026-09-02T04:34:17Z)
• SkillForge: Evolving Verifiable Skills for Reinforcement Learning Agents [17.565432600589414]
  Large language model (LLM) agents are trained with reinforcement learning (RL) for complex decision-making tasks.<n>Recent skill-based approaches, such as SkillRL, attempt to address this issue by extracting skills from raw trajectories.<n>We propose SkillForge, a framework for continuous skill evolution that enables skills to be verified and refined through environment interaction.
  arXiv  Detail & Related papers  (2026-08-25T15:52:10Z)
• Coalition-Aware Skill Reliability for Self-Evolving Agents [49.43736235959317]
  We investigate whether accumulated skills in an agent's skill bank actually make positive mechanistic contributions.<n>We show that coalition pollution, where bank-level gains conceal negative coalition-level skill contributions, and cross-domain utility reversal, where source-beneficial skills reverse their effects after transfer.<n>These findings motivate two reliability interventions: coalition-aware skill selection during skill accumulation and label-free skill masking after transfer.
  arXiv  Detail & Related papers  (2026-08-23T21:35:22Z)
• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• SkillMaster: Toward Autonomous Skill Mastery in LLM Agents [27.651128308229378]
  We propose SkillMaster, a training framework that teaches agents to create new skills, refine existing skills, and select accumulated skills during task solving.<n>First, we train agents through trajectory-informed skill review, teaching agents to propose, update, or retain skills based on evidence from completed episodes.<n>Second, each candidate skill edit is designed to be evaluated by its counterfactual utility on related probe tasks, providing a direct learning signal for training skill-editing decisions.<n>Third, we introduce DualAdv-GRPO, which separately estimates advantages for task-solving actions and skill-editing decisions, stabilizing joint training across task solving and
  arXiv  Detail & Related papers  (2026-05-09T05:03:00Z)
• SkillOS: Learning Skill Curation for Self-Evolving Agents [67.94374107466957]
  We propose SkillOS, an experience-driven RL training recipe for learning skill curation in self-evolving agents.<n>SkillOS pairs a frozen agent executor that retrieves and applies skills with a trainable skill curator that updates an external SkillRepo from accumulated experience.<n>We show that SkillOS consistently outperforms memory-free and strong memory-based baselines in both effectiveness and efficiency.
  arXiv  Detail & Related papers  (2026-05-07T17:31:50Z)
• EvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification [85.3692584167951]
  Anthropic proposes the concept of skills for LLM agents to tackle multi-step professional tasks.<n>A tool is a single, self-contained function, whereas a skill is a structured bundle of interdependent multi-file artifacts.<n>EvoSkills is a self-evolving skills framework that enables agents to autonomously construct complex, multi-file skill packages.
  arXiv  Detail & Related papers  (2026-04-02T06:43:20Z)
• Dynamic Dual-Granularity Skill Bank for Agentic RL [34.161117844675324]
  D2Skill is a dynamic dual-granularity skill bank for agentic reinforcement learning.<n>It organizes reusable experience into task skills for high-level guidance and step skills for fine-grained decision support and error correction.
  arXiv  Detail & Related papers  (2026-03-30T17:32:11Z)
• Reinforcement Learning for Self-Improving Agent with Skill Library [14.717149089634718]
  Large Language Model (LLM)-based agents have demonstrated remarkable capabilities in complex reasoning and multi-turn interactions.<n>One promising approach is implementing skill libraries that allow agents to learn, validate, and apply new skills.<n>We propose a Reinforcement Learning (RL)-based approach to enhance agents' self-improvement capabilities with a skill library.
  arXiv  Detail & Related papers  (2025-12-18T21:58:19Z)
• PillagerBench: Benchmarking LLM-Based Agents in Competitive Minecraft Team Environments [48.892997022500765]
  We introduce PillagerBench, a framework for evaluating multi-agent systems in real-time competitive team-vs-team scenarios in Minecraft.<n>We also propose TactiCrafter, an LLM-based multi-agent system that facilitates teamwork through human-readable tactics.<n>Our evaluation demonstrates that TactiCrafter outperforms baseline approaches and showcases adaptive learning through self-play.
  arXiv  Detail & Related papers  (2025-09-07T22:51:12Z)
• Who is a Better Player: LLM against LLM [53.46608216197315]
  We propose an adversarial benchmarking framework to assess the comprehensive performance of Large Language Models (LLMs) through board games competition.<n>We introduce Qi Town, a specialized evaluation platform that supports 5 widely played games and involves 20 LLM-driven players.
  arXiv  Detail & Related papers  (2025-08-05T06:41:47Z)
• Multi-Agent Training for Pommerman: Curriculum Learning and Population-based Self-Play Approach [11.740631954398292]
  Pommerman is an ideal benchmark for multi-agent training, providing a battleground for two teams with communication capabilities among allied agents.<n>This study introduces a system designed to train multi-agent systems to play Pommerman using a combination of curriculum learning and population-based self-play.
  arXiv  Detail & Related papers  (2024-06-30T11:14:29Z)
• Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game [37.69298376616128]
  We develop strategic language agents that generate flexible language actions and possess strong decision-making abilities.<n>To mitigate the intrinsic bias in language actions, our agents use an LLM to perform deductive reasoning and generate a diverse set of action candidates.<n>Experiments show that our agents overcome the intrinsic bias and outperform existing LLM-based agents in the Werewolf game.
  arXiv  Detail & Related papers  (2023-10-29T09:02:57Z)
• AgentBench: Evaluating LLMs as Agents [99.12825098528212]
  Large Language Model (LLM) as agents has been widely acknowledged recently.<n>We present AgentBench, a benchmark that consists of 8 distinct environments to assess LLM-as-Agent's reasoning and decision-making abilities.
  arXiv  Detail & Related papers  (2023-08-07T16:08:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.