Learning to Self-Evolve

• URL: http://arxiv.org/abs/2603.18620v1
• Date: Thu, 19 Mar 2026 08:41:24 GMT
• Title: Learning to Self-Evolve
• Abstract summary: We introduce Learning to Self-Evolve (LSE), a reinforcement learning framework that trains large language models (LLMs) to improve their own contexts at test time.<n>LSE reduces the multi-step evolution problem to a single-step RL objective, where each context edit is rewarded by the improvement in downstream performance.<n>Our results highlight the effectiveness of treating self-evolution as a learnable skill.
• Score: 37.39989112050656
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Learning to Self-Evolve (LSE), a reinforcement learning framework that trains large language models (LLMs) to improve their own contexts at test time. We situate LSE in the setting of test-time self-evolution, where a model iteratively refines its context from feedback on seen problems to perform better on new ones. Existing approaches rely entirely on the inherent reasoning ability of the model and never explicitly train it for this task. LSE reduces the multi-step evolution problem to a single-step RL objective, where each context edit is rewarded by the improvement in downstream performance. We pair this objective with a tree-guided evolution loop. On Text-to-SQL generation (BIRD) and general question answering (MMLU-Redux), a 4B-parameter model trained with LSE outperforms self-evolving policies powered by GPT-5 and Claude Sonnet 4.5, as well as prompt optimization methods including GEPA and TextGrad, and transfers to guide other models without additional training. Our results highlight the effectiveness of treating self-evolution as a learnable skill.

Related papers

• DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution [30.082997083095847]
  Large language models (LLMs) cannot retain post-deployment experience without parameter updates.<n>We introduce DIVE, a diversity-driven framework that enables frozen LLMs to improve by evolving from task experience and verifier feedback.
  arXiv  Detail & Related papers  (2026-08-12T18:06:41Z)
• Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning [51.17262419591891]
  Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains.<n>We present MetaEvolve, a framework designed to develop these meta-skills via a data synthesis pipeline, evolution-aware reinforcement learning, and inference-time evolutionary search.
  arXiv  Detail & Related papers  (2026-07-24T04:35:29Z)
• SEIF: Self-Evolving Reinforcement Learning for Instruction Following [53.280277743734096]
  We propose SEIF (Self-Evolving Reinforcement Learning for Instruction Following), a self-evolving framework for enhancing the instruction-following ability of large language models (LLMs)<n>SEIF forms a closed self-evolution loop that improves the model's instruction-following ability, where instruction difficulty evolution and model capability evolution reinforce each other.<n> Experiments across multiple model scales and architectures show that SEIF consistently improves instruction-following performance, suggesting strong generality.
  arXiv  Detail & Related papers  (2026-05-08T09:13:12Z)
• ThetaEvolve: Test-time Learning on Open Problems [110.5756538358217]
  We introduce ThetaEvolve, an open-source framework that simplifies and extends AlphaEvolve to efficiently scale both in-context learning and Reinforcement Learning (RL) at test time.<n>We find that ThetaEvolve with RL at test-time consistently outperforms inference-only baselines.
  arXiv  Detail & Related papers  (2025-11-28T18:58:14Z)
• Evolving LLMs' Self-Refinement Capability via Synergistic Training-Inference Optimization [53.93621974137829]
  Self-Refinement refers to a model's ability to revise its own responses to produce improved outputs.<n>EVOLVE is a framework for eliciting and tracking the evolution of Self-Refinement through iterative training.<n>We demonstrate the potential of leveraging Self-Refinement to achieve broader Self-Improvement of intrinsic model abilities.
  arXiv  Detail & Related papers  (2025-02-08T15:21:55Z)
• A Survey on Self-Evolution of Large Language Models [116.54238664264928]
  Large language models (LLMs) have significantly advanced in various fields and intelligent agent applications. To address this issue, self-evolution approaches that enable LLMs to autonomously acquire, refine, and learn from experiences generated by the model itself are rapidly growing.
  arXiv  Detail & Related papers  (2024-04-22T17:43:23Z)
• SELF: Self-Evolution with Language Feedback [68.6673019284853]
  'SELF' (Self-Evolution with Language Feedback) is a novel approach to advance large language models. It enables LLMs to self-improve through self-reflection, akin to human learning processes. Our experiments in mathematics and general tasks demonstrate that SELF can enhance the capabilities of LLMs without human intervention.
  arXiv  Detail & Related papers  (2023-10-01T00:52:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.