Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain

• URL: http://arxiv.org/abs/2603.02218v1
• Date: Tue, 10 Feb 2026 08:12:09 GMT
• Title: Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain
• Abstract summary: Large language models (LLMs) make it plausible to build systems that improve through self-evolving loops.<n>We show that sustainable self-evolution requires a self-synthesised data pipeline with learnable information that increases across iterations.
• Score: 22.77669491242655
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) make it plausible to build systems that improve through self-evolving loops, but many existing proposals are better understood as self-play and often plateau quickly. A central failure mode is that the loop synthesises more data without increasing learnable information for the next iteration. Through experiments on a self-play coding task, we reveal that sustainable self-evolution requires a self-synthesised data pipeline with learnable information that increases across iterations. We identify triadic roles that self-evolving LLMs play: the Proposer, which generates tasks; the Solver, which attempts solutions; and the Verifier, which provides training signals, and we identify three system designs that jointly target learnable information gain from this triadic roles perspective. Asymmetric co-evolution closes a weak-to-strong-to-weak loop across roles. Capacity growth expands parameter and inference-time budgets to match rising learnable information. Proactive information seeking introduces external context and new task sources that prevent saturation. Together, these modules provide a measurable, system-level path from brittle self-play dynamics to sustained self-evolution.

Related papers

• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• Self-evolving LLM agents with in-distribution Optimization [60.05867547965365]
  Large Language Models (LLMs) have recently emerged as powerful controllers for interactive agents in complex environments.<n>We propose Q-Evolve, a self-evolving framework for LLM agents that unifies automatic process-reward labeling and policy learning.<n>We evaluate our method on AlfWorld, WebShop, and ScienceWorld, showing Q-Evolve outperforms strong baselines in sample efficiency, robustness, and overall task performance.
  arXiv  Detail & Related papers  (2026-06-05T15:09:52Z)
• RISE: Reliable Improvement in Self-Evolving Vision-Language Models [22.16641554921884]
  Vision-language models (VLMs) have achieved strong multimodal reasoning capabilities.<n>VLMs still rely heavily on large-scale human-constructed supervision for post-training.<n>We propose textbfRISE, a reliable self-evolving framework for vision-language models.
  arXiv  Detail & Related papers  (2026-05-20T08:57:57Z)
• Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolution [57.04954428528267]
  We introduce Ace-Skill, a co-evolutionary framework that jointly optimize rollout allocation and knowledge organization for self-evolving multimodal agents.<n>Ace-Skill combines aprioritized sampler with lazy-decay proficiency tracking to focus rollouts on informative and insufficiently mastered samples.<n>By improving sampling and organization together, Ace-Skill turns self-evolution into a virtuous cycle in which more informative rollouts produce higher-quality knowledge.
  arXiv  Detail & Related papers  (2026-05-09T11:12:54Z)
• Yunjue Agent Tech Report: A Fully Reproducible, Zero-Start In-Situ Self-Evolving Agent System for Open-Ended Tasks [10.622439192272527]
  Conventional agent systems struggle in open-ended environments where task distributions continuously drift and external supervision is scarce.<n>We propose the In-Situ Self-Evolving paradigm, which treats sequential task interactions as a continuous stream of experience.<n>Within this framework, we develop Yunjue Agent, a system that iteratively synthesizes, optimize, and reuses tools to navigate emerging challenges.
  arXiv  Detail & Related papers  (2026-01-26T07:27:47Z)
• SPELL: Self-Play Reinforcement Learning for evolving Long-Context Language Models [79.01078135582127]
  SPELL enables scalable, label-free optimization for long-context reasoning.<n>We introduce an automated curriculum that gradually increases document length and a reward function that adapts question difficulty to the model's evolving capabilities.
  arXiv  Detail & Related papers  (2025-09-28T13:08:10Z)
• Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing [4.874077691069634]
  Retrieval Augmented Generation (RAG) has shown strong capability in enhancing language models' knowledge and reducing AI generative hallucinations.<n>Current multi-round RAG systems may continue searching even when enough information has already been retrieved.<n>This paper introduces a new framework, SIM-RAG, to explicitly enhance RAG systems' self-awareness and multi-round retrieval capabilities.
  arXiv  Detail & Related papers  (2025-05-05T17:39:35Z)
• A Survey on Self-Evolution of Large Language Models [116.54238664264928]
  Large language models (LLMs) have significantly advanced in various fields and intelligent agent applications. To address this issue, self-evolution approaches that enable LLMs to autonomously acquire, refine, and learn from experiences generated by the model itself are rapidly growing.
  arXiv  Detail & Related papers  (2024-04-22T17:43:23Z)
• SELF: Self-Evolution with Language Feedback [68.6673019284853]
  'SELF' (Self-Evolution with Language Feedback) is a novel approach to advance large language models. It enables LLMs to self-improve through self-reflection, akin to human learning processes. Our experiments in mathematics and general tasks demonstrate that SELF can enhance the capabilities of LLMs without human intervention.
  arXiv  Detail & Related papers  (2023-10-01T00:52:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.