SEIF: Self-Evolving Reinforcement Learning for Instruction Following

• URL: http://arxiv.org/abs/2605.07465v1
• Date: Fri, 08 May 2026 09:13:12 GMT
• Title: SEIF: Self-Evolving Reinforcement Learning for Instruction Following
• Abstract summary: We propose SEIF (Self-Evolving Reinforcement Learning for Instruction Following), a self-evolving framework for enhancing the instruction-following ability of large language models (LLMs)<n>SEIF forms a closed self-evolution loop that improves the model's instruction-following ability, where instruction difficulty evolution and model capability evolution reinforce each other.<n> Experiments across multiple model scales and architectures show that SEIF consistently improves instruction-following performance, suggesting strong generality.
• Score: 53.280277743734096
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instruction following is a fundamental capability of large language models (LLMs), yet continuously improving this capability remains challenging. Existing methods typically rely either on costly external supervision from humans or strong teacher models, or on self-play training with static-difficulty instructions that cannot evolve as the model's capabilities improve. To address these limitations, we propose SEIF (Self-Evolving Reinforcement Learning for Instruction Following), a self-evolving framework for enhancing the instruction-following ability of LLMs. SEIF forms a closed self-evolution loop that improves the model's instruction-following ability, where instruction difficulty evolution and model capability evolution reinforce each other. SEIF consists of four roles: an Instructor that generates increasingly challenging instructions, a Filter that removes conflicting or invalid instructions to ensure data quality, a Follower that learns to follow evolved instructions, and a Judger that provides reward signals for reinforcement learning. The Instructor and Follower are alternately trained and co-evolve throughout the process. Experiments across multiple model scales and architectures show that SEIF consistently improves instruction-following performance, suggesting strong generality. Further analyses reveal the sources of improvement and identify an effective training strategy for self-evolution on open-ended tasks: sufficient early-stage training to build a solid foundation, followed by moderate late-stage training to mitigate overfitting and achieve better final performance. The code and data are publicly available at https://github.com/Rainier-rq1/SEIF.

Related papers

• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• Diffusion Guidance Is a Controllable Policy Improvement Operator [98.11511661904618]
  CFGRL is trained with the simplicity of supervised learning, yet can further improve on the policies in the data.<n>On offline RL tasks, we observe a reliable trend -- increased guidance weighting leads to increased performance.
  arXiv  Detail & Related papers  (2025-05-29T14:06:50Z)
• SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data [65.56911325914582]
  We propose Self-play Reinforcement Learning (SeRL) to bootstrap Large Language Models (LLMs) training with limited initial data.<n>The proposed SeRL yields results superior to its counterparts and achieves performance on par with those obtained by high-quality data with verifiable rewards.
  arXiv  Detail & Related papers  (2025-05-25T13:28:04Z)
• LLM Post-Training: A Deep Dive into Reasoning Large Language Models [131.10969986056]
  Large Language Models (LLMs) have transformed the natural language processing landscape and brought to life diverse applications.<n>Post-training methods enable LLMs to refine their knowledge, improve reasoning, enhance factual accuracy, and align more effectively with user intents and ethical considerations.
  arXiv  Detail & Related papers  (2025-02-28T18:59:54Z)
• Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuning [30.82220015525281]
  Mosaic Instruction Tuning (Mosaic-IT) is a human/model-free compositional data synthesis method.<n>Our evaluations demonstrate a superior performance and training efficiency of Mosaic-IT.
  arXiv  Detail & Related papers  (2024-05-22T04:08:20Z)
• Improving the Robustness of Large Language Models via Consistency Alignment [36.24876571343749]
  Large language models (LLMs) have shown tremendous success in following user instructions and generating helpful responses. LLMs may generate significantly inconsistent responses due to minor changes in the verbalized instructions. We propose a two-stage training framework consisting of instruction-augmented supervised fine-tuning and consistency alignment training.
  arXiv  Detail & Related papers  (2024-03-21T08:21:12Z)
• SELF: Self-Evolution with Language Feedback [68.6673019284853]
  'SELF' (Self-Evolution with Language Feedback) is a novel approach to advance large language models. It enables LLMs to self-improve through self-reflection, akin to human learning processes. Our experiments in mathematics and general tasks demonstrate that SELF can enhance the capabilities of LLMs without human intervention.
  arXiv  Detail & Related papers  (2023-10-01T00:52:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.