Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds

• URL: http://arxiv.org/abs/2608.02636v1
• Date: Fri, 31 Jul 2026 04:02:51 GMT
• Title: Rethinking Self-Evolving Agent Skills: Feedback Dynamics over Multiple Rounds
• Abstract summary: Self-evolving skill systems promise to improve agents by turning execution feedback into persistent skill updates without changing the underlying model.<n>We present a controlled evaluation framework across five benchmarks and three models.<n>Overall, persistent skill self-evolution is better understood as sparse, validation-filtered search with model- and benchmark-dependent returns.
• Score: 43.5891705224868
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-evolving skill systems promise to improve agents by turning execution feedback into persistent skill updates without changing the underlying model. Yet it remains unclear when further evolution helps, how successful and failed trajectories shape revision, and whether extra test-time computation can recover the same gains. To address these questions, we present a controlled evaluation framework across five benchmarks and three models. Our primary study contains 42 feedback runs across 14 supported model-benchmark settings. Within each setting, we hold the executor and optimizer configuration, revision procedure, validation rule, and round budget fixed, while varying only the feedback shown to the optimizer: successes and failures (Normal), failures only, or successes only. Evolution is sparse: only 55 of 388 candidates establish byte-distinct validation bests. Validation-based selection chooses an evolved skill in 11 of 14 settings, nine of which improve released-test performance. All 11 selections come from feedback conditions that include failed trajectories, although the relative ranking of Normal and Fail-only varies across settings. Validation and downstream evaluations on test, robustness, and transfer sometimes favor different feedback views. A broader SearchQA analysis covering eight models shows similarly sparse, feedback-dependent dynamics. In the GPT-5.5 test-time-scaling controls, oracle Parallel Sampling comes within 0.43 points of the evolved SearchQA skill but remains 30.96 points behind on SpreadsheetBench; Sequential Refinement recovers neither gain. Overall, persistent skill self-evolution is better understood as sparse, validation-filtered search with model- and benchmark-dependent returns, rather than steady improvement from additional rounds. The implementation is available at https://github.com/HKUST-KnowComp/rethinkskill.

Related papers

• TRACE: A Self-Evolving Skill Bank for Consistent, Limit-Aware LLM Agents [41.82249291285459]
  TRACE iteratively improves a skill-based agent's behavioral knowledge without modifying model weights.<n>On GPT-5.5, TRACE improves consistency (Pass3) by 34.6 points, from 59.9% to 94.5%, while shrinking the gap between potential and reliable performance to just 4.0 points.<n>These results show that TRACE converts high model potential into stable, consistent performance gain.
  arXiv  Detail & Related papers  (2026-08-24T04:36:03Z)
• Write, Execute, Refine: From Skill Followers to Skill Optimizers via Reinforcement Learning from Execution Feedback [52.78692582507693]
  Expert-written natural language skills can improve tool-using agents, yet agent-authored skills perform 8-11 points worse than using no skill.<n>We study how to organize execution experience from intermediate skills into training states for an executor.
  arXiv  Detail & Related papers  (2026-08-18T09:52:48Z)
• PACE-Bench: Benchmarking Physics Adaptation via Code Evolution in Dynamic Environments [8.552837617531766]
  PACE-Bench is a simulator-grounded benchmark of 144 source-to-target adaptation pairs across six physics domains.<n>We show that simulator-grounded reflection is more reliable than self-revision.<n>Even revealing exact physical changes does not raise the performance ceiling.
  arXiv  Detail & Related papers  (2026-08-14T16:25:43Z)
• DarwinX: Evolving Agent Harnesses Through Natural Selection [48.64258219266629]
  DarwinX treats self-evolution as selection over a population of harnesses with the model frozen.<n>General agent competence, not benchmark-specific patches, so it survives changes of task, verifier, and base model.
  arXiv  Detail & Related papers  (2026-07-31T05:34:02Z)
• RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement [27.641105176473136]
  Existing benchmarks entangle research decisions with optimization, serving, evaluation, and systems implementation.<n>We introduce RSIBench-Data, a benchmark of LLM agents as data-centric researchers with a fixed post-training stack.<n>We evaluate four frontier agents on six benchmarks across software engineering, terminal use, scientific question answering, and mathematics.
  arXiv  Detail & Related papers  (2026-07-28T15:46:41Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• From Failing to Passing: Evolving Natural Language Prompt Optimization Rules for LLM Code Generation [11.454560846406698]
  We introduce a search-based approach that identifies and evolves a set of natural language transformation rules.<n>We then proposeFIX, a staged repair pipeline that combines the evolved transformation rules with execution-feedback repair.<n>Our results show that DualFix recovers up to 30% of baseline failures and fixes 3-5 times more failing cases than Self-Fix.
  arXiv  Detail & Related papers  (2026-07-06T14:10:56Z)
• Regression Accumulation in Multi-Turn LLM Programming Conversations [25.94870617800846]
  regression accumulation appears across all six models.<n>Cross-Turn Conflict, where later code conflicts with earlier requirements, is the main failure class.<n> Verification Gate is the only strategy that consistently improves all models.
  arXiv  Detail & Related papers  (2026-07-02T08:15:40Z)
• REVES: REvision and VErification--Augmented Training for Test-Time Scaling [53.197756110943395]
  We propose a two-stage iterative framework that alternates between online data/prompt augmentation and policy optimization.<n>We observe gains of +6.5 points over the RL baseline and +4.0 points over standard multi-turn training.
  arXiv  Detail & Related papers  (2026-06-17T10:37:23Z)
• GRASP: Gated Regression-Aware Skill Proposer for Self-Improving LLM Agents [17.26555663132631]
  GRASP (Gated Regression-Aware Skill Proposer) treats agent improvement as a sequence of edits to a bounded skill library.<n>We evaluate GRASP across five base models on clinical benchmarks.
  arXiv  Detail & Related papers  (2026-05-28T09:30:09Z)
• EvoCode-Bench: Evaluating Coding Agents in Multi-Turn Iterative Interactions [9.297494684604763]
  EvoCode-Bench is a benchmark of 26 stateful coding tasks and 227 evaluated rounds.<n>Each task preserves the agent's workspace for 5-15 rounds, states requirements through observable behavior.<n>We evaluate 13 coding agents with two metrics: MT@4, a four-attempt fail-stop multi-round score, and SR, a single-round score from a reference-completed prior state.
  arXiv  Detail & Related papers  (2026-05-22T18:17:28Z)
• ClawArena: Benchmarking AI Agents in Evolving Information Environments [61.664633997138004]
  ClawArena is a benchmark for evaluating AI agents in evolving information environments.<n>Each scenario maintains a complete hidden ground truth while exposing the agent only to noisy, partial, and sometimes contradictory traces.<n> evaluation is organized around three coupled challenges: multi-source conflict reasoning, dynamic belief revision, and implicit personalization.
  arXiv  Detail & Related papers  (2026-04-05T17:55:23Z)
• Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models [94.68358825189738]
  We propose a post-training framework for verifiable domains that jointly optimize prediction accuracy and reasoning quality.<n>We evaluate VPS on chess, a controlled testbed where reasoning steps can be deterministically verified against engine signals.<n>VPS preserves accuracy while significantly improving reasoning quality, reducing win-rate error by up to 30% and restoring consistency to near saturation.
  arXiv  Detail & Related papers  (2026-04-03T15:19:46Z)
• Test-time Recursive Thinking: Self-Improvement without External Feedback [120.80790108733942]
  Test-time Recursive Thinking (TRT) is an iterative self-improvement framework.<n>Open-source models reach 100% accuracy on AIME-25/24, and on LiveCodeBench's most difficult problems, closed-source models improve by 10.4-14.8 percentage points without external feedback.
  arXiv  Detail & Related papers  (2026-02-03T04:37:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.