DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution

• URL: http://arxiv.org/abs/2608.12486v1
• Date: Wed, 12 Aug 2026 18:06:41 GMT
• Title: DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution
• Abstract summary: Large language models (LLMs) cannot retain post-deployment experience without parameter updates.<n>We introduce DIVE, a diversity-driven framework that enables frozen LLMs to improve by evolving from task experience and verifier feedback.
• Score: 30.082997083095847
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) cannot retain post-deployment experience without parameter updates. We introduce DIVE, a diversity-driven framework that enables frozen LLMs to improve by evolving persistent natural-language skills from task experience and verifier feedback. These skills encode reusable reasoning procedures, verification strategies, common failure modes, and output constraints and are both executed and revised by the same underlying model without access to a teacher model. Since natural-language skill evolution is a stochastic, non-convex search process, optimizing a single skill trajectory can overfit to sampled experience or converge to a suboptimal solution. DIVE mitigates this optimization variance by independently evolving multiple skill populations from bootstrapped experience, adaptively refining them through diverse transformations, and jointly selecting a complementary set of skills. Across six mathematical and logical reasoning tasks and multiple model families, DIVE consistently outperforms existing reasoning methods, prompt-optimization approaches, skill-development frameworks, and memory-based baselines. It achieves rapid self-improvement from accumulated experience, obtaining substantially larger performance gains with fewer rollouts than parameter-based methods such as SFT and GRPO, and prompt optimization with GEPA. Further, the resulting skills transfer across model scales and families, enabling smaller models such as GPT-5-nano to match or outperform larger counterparts, i.e., GPT-5, under conventional prompting. These results establish diversity-driven skill evolution as an effective, interpretable, and parameter-free approach to LLM self-improvement.

Related papers

• Self-Evolving Embodied Agents via Skill-Harness Evolution [53.307052568223945]
  Embodied agents are increasingly built as systems around foundation models.<n>While supervised fine-tuning and reinforcement learning can adapt agents to new environments, they require additional data, rewards, and training runs.<n>We propose SHAPER, a self-evolving framework for train-free embodied adaptation that keeps model parameters frozen.
  arXiv  Detail & Related papers  (2026-08-11T18:55:58Z)
• Self-Improving Large Language Models via Progressive Experience Evolution [52.03479747358687]
  We propose textbfSPEE (textbfSelf-textbfProgressive textbfExperience textbfEvolution), a unified post-training framework that sequentially performs explicit experience evolution followed by implicit policy optimization.<n> Experiments on five mathematical reasoning benchmarks demonstrate that SPEE consistently outperforms both test-time and training-time self-evolution baselines across three model scales.
  arXiv  Detail & Related papers  (2026-08-03T12:27:32Z)
• Rethinking Self-Evolution: A Constrained Exploration-Exploitation Process for Mitigating Skill Overfitting [20.811638832981178]
  Large language model (LLM) agents need to accumulate and reuse experience from past interactions.<n>Data-driven skill optimization is prone to overfitting to the limited trajectories collected from real environments.<n>We propose SkillBoost, a three-stage framework that mitigates both risks.
  arXiv  Detail & Related papers  (2026-07-29T09:05:40Z)
• Skill-R1: Agent Skill Evolution via Reinforcement Learning [84.35984979949502]
  Skill-R1 is a reinforcement learning framework for instance-level recurrent skill optimization from verifiable rewards.<n>It preserves black-box compatibility with both open- and closed-source models while making adaptation substantially cheaper than model-level updates.
  arXiv  Detail & Related papers  (2026-05-10T06:19:15Z)
• DIMoE-Adapters: Dynamic Expert Evolution for Continual Learning in Vision-Language Models [63.58458696122293]
  Continual learning enables vision-language models to accumulate knowledge and adapt to evolving tasks without retraining from scratch.<n>Most existing methods rely on fixed architectures with statically allocated parameters, which limits adaptation to new domains and aggravates catastrophic forgetting.<n>We propose DIMoE-Adapters, a Dynamic Incremental Mixture-of-Experts Adapters framework that introduces a dynamic expert evolution paradigm to balance stability and plasticity.
  arXiv  Detail & Related papers  (2026-05-08T09:32:05Z)
• Skills-Coach: A Self-Evolving Skill Optimizer via Training-Free GRPO [31.696187731033334]
  Skills-Coach is an automated framework designed to enhance the self-evolution of skills within Large Language Model (LLM)-based agents.<n>The framework comprises four core modules: a Diverse Task Generation Module that creates a comprehensive test suite for various skills; a Lightweight Optimization Module dedicated to optimizing skill prompts and their corresponding code.<n>To validate its efficacy, we introduce Skill-X, a comprehensive benchmark dataset consisting of 48 diverse skills.
  arXiv  Detail & Related papers  (2026-04-30T06:39:37Z)
• Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models [83.8639566087953]
  We propose a direct retrieval-augmented optimization framework, named DRO, that enables end-to-end training of two key components.<n>DRO alternates between two phases: (i) document permutation estimation and (ii) re-weighted, progressively improving RAG components.<n>Our theoretical analysis reveals that DRO is analogous to policy-gradient methods in reinforcement learning.
  arXiv  Detail & Related papers  (2025-05-05T23:54:53Z)
• Improving Retrospective Language Agents via Joint Policy Gradient Optimization [57.35348425288859]
  RetroAct is a framework that jointly optimize both task-planning and self-reflective evolution capabilities in language agents.<n>We develop a two-stage joint optimization process that integrates imitation learning and reinforcement learning.<n>We conduct extensive experiments across various testing environments, demonstrating RetroAct has substantial improvements in task performance and decision-making processes.
  arXiv  Detail & Related papers  (2025-03-03T12:54:54Z)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
  Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and. Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting. LLMs to downstream tasks. We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
  arXiv  Detail & Related papers  (2024-09-30T10:48:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.