What Do Evolutionary Coding Agents Evolve?

• URL: http://arxiv.org/abs/2605.20086v1
• Date: Tue, 19 May 2026 16:41:45 GMT
• Title: What Do Evolutionary Coding Agents Evolve?
• Abstract summary: EvoTrace is a dataset of evolutionary coding traces spanning four evolutionary frameworks, reasoning and non-reasoning models, and 16 tasks across mathematics and algorithm design.<n>We analyze these traces with EvoReplay, a replay-based methodology that reconstructs the local search states behind high-scoring solutions.<n>Across EvoTrace, most score gains come from a small subset of these edit types.
• Score: 46.561777689365165
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work pairs LLMs with evolutionary search to iteratively generate, modify, and select code using task-specific feedback. These systems have produced strong results in mathematical discovery and algorithm design, yet a fundamental question remains: what do they actually evolve? Progress is typically summarized by the best score a run reaches under a task-specific evaluator, but that score can reflect several different mechanisms: new algorithmic structure, re-tuning an existing strategy, recombining ideas already in the model's internal knowledge, or overfitting to the evaluator. Distinguishing these mechanisms requires inspecting the search process itself, not only its final outcome. We introduce EvoTrace, a dataset of evolutionary coding traces spanning four evolutionary frameworks, reasoning and non-reasoning models, and 16 tasks across mathematics and algorithm design. To analyze these traces, we develop EvoReplay, a replay-based methodology that reconstructs the local search states behind high-scoring solutions and tests controlled interventions, including adjusting constants, removing program components and substituting models or prompting contexts. We annotate every code edit in EvoTrace with one of nine recurring edit types using an LLM-as-judge pipeline validated against blind human re-annotation. Across EvoTrace, most score gains come from a small subset of these edit types. We further find a deterministic cycling pattern: about 30% of code lines added during search are byte-identical re-introductions of previously-deleted lines, present throughout nearly every run. These results show that benchmark gains in evolutionary coding agents can arise from qualitatively different mechanisms, only some of which correspond to new algorithmic structure. EvoTrace enables more diagnostic evaluation of evolutionary coding agents beyond final benchmark scores.

Related papers

• EvoMem: Memory-Augmented Evolution for Code Optimization [7.069994048454947]
  We introduce EvoMem, a persistent memory architecture for evolutionary program search.<n>EvoMem converts successful mutation events into structured, task-aware advice for future runs.<n>Our experiments show positive average improvements in target metrics or search speed for most evaluated settings.
  arXiv  Detail & Related papers  (2026-08-11T11:03:32Z)
• Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning [51.17262419591891]
  Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains.<n>We present MetaEvolve, a framework designed to develop these meta-skills via a data synthesis pipeline, evolution-aware reinforcement learning, and inference-time evolutionary search.
  arXiv  Detail & Related papers  (2026-07-24T04:35:29Z)
• Rethinking the Evaluation of Harness Evolution for Agents [83.07258924910069]
  Existing harness evolution methods use unit test cases to search for harness configurations and then report final performance on the same public benchmark.<n>This protocol raises two fundamental concerns.<n>As in agentic test-time scaling, it should therefore be compared with simple task-level search baselines under matched feedback and inference budgets.
  arXiv  Detail & Related papers  (2026-07-14T00:18:42Z)
• AlgoEvolve: LLM-driven Meta-evolution of Algorithmic Trading Programs [11.243181562664802]
  We present AlgoEvolve, an evolutionary framework that generates, evaluates, and iteratively improves executable trading strategies.<n>Across multiple experiments, the system exhibits emergent regime-adaptive strategy logic, including autonomous shifts in trading rules.
  arXiv  Detail & Related papers  (2026-06-24T10:05:03Z)
• Self-Improving Language Models with Bidirectional Evolutionary Search [93.21736992722835]
  Bidirectional Evolutionary Search (BES) is a search framework that couples forward candidate evolution with backward goal decomposition.<n>We provide theoretical motivation showing that candidates generated by expansion-only search are confined to a narrow entropy shell while evolutionary operators can escape it.<n> Experiments show that on challenging post-training tasks where mainstream post-training algorithms fail to improve, BES enables consistent gains.
  arXiv  Detail & Related papers  (2026-05-27T17:59:15Z)
• EvoForest: A Novel Machine-Learning Paradigm via Open-Ended Evolution of Computational Graphs [2.7010154811483162]
  We present EvoForest, a hybrid neuro-symbolic system for end-to-end open-ended evolution of computation.<n>Rather than merely generating features, EvoForest jointly evolves reusable computational structure, callable function families, and trainable low-dimensional continuous components.<n>For each graph configuration, EvoForest evaluates the discovered computation and uses a lightweight Ridge-based readout to score the resulting representation.
  arXiv  Detail & Related papers  (2026-03-26T00:07:45Z)
• EvoX: Meta-Evolution for Automated Discovery [115.89434419482797]
  EvoX is an adaptive evolution method that optimize its own evolution process.<n>It continuously updates how prior solutions are selected and varied based on progress.<n>It outperforms existing AI-driven evolutionary methods including AlphaEvolve, OpenEvolve, GEPA, and ShinkaEvolve on the majority of tasks.
  arXiv  Detail & Related papers  (2026-02-26T18:54:41Z)
• Self-EvolveRec: Self-Evolving Recommender Systems with LLM-based Directional Feedback [21.326241484461587]
  Self-EvolveRec is a novel framework that establishes a directional feedback loop.<n>It significantly outperforms state-of-the-art NAS and LLM-driven code evolution baselines in both recommendation performance and user satisfaction.
  arXiv  Detail & Related papers  (2026-02-13T04:38:32Z)
• DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution [28.737322041874293]
  LLM-driven evolutionary systems have shown promise for automated science discovery.<n>Existing approaches such as AlphaEvolve rely on full-code histories that are context-inefficient.<n>We propose DeltaEvolve, a momentum-driven evolutionary framework that replaces full-code history with structured semantic delta.
  arXiv  Detail & Related papers  (2026-02-02T23:47:54Z)
• GigaEvo: An Open Source Optimization Framework Powered By LLMs And Evolution Algorithms [7.228213026504935]
  GigaEvo is an open-source framework that enables researchers to study and experiment with hybrid LLM-evolution approaches.<n>We provide detailed descriptions of system architecture, implementation decisions, and experimental methodology to support further research.
  arXiv  Detail & Related papers  (2025-11-17T14:44:47Z)
• Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking [54.43083499412643]
  Test-time algorithms that combine the generative power of language models with process verifiers offer a promising lever for eliciting new reasoning capabilities.<n>We introduce a new process-guided test-time sampling algorithm, VGB, which uses theoretically grounded backtracking to achieve provably better robustness to verifier errors.
  arXiv  Detail & Related papers  (2025-10-03T16:21:14Z)
• Phylogeny-informed fitness estimation [58.720142291102135]
  We propose phylogeny-informed fitness estimation, which exploits a population's phylogeny to estimate fitness evaluations. Our results indicate that phylogeny-informed fitness estimation can mitigate the drawbacks of down-sampled lexicase. This work serves as an initial step toward improving evolutionary algorithms by exploiting runtime phylogenetic analysis.
  arXiv  Detail & Related papers  (2023-06-06T19:05:01Z)
• Autoregressive Search Engines: Generating Substrings as Document Identifiers [53.0729058170278]
  Autoregressive language models are emerging as the de-facto standard for generating answers. Previous work has explored ways to partition the search space into hierarchical structures. In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
  arXiv  Detail & Related papers  (2022-04-22T10:45:01Z)
• AutoML-Zero: Evolving Machine Learning Algorithms From Scratch [76.83052807776276]
  We show that it is possible to automatically discover complete machine learning algorithms just using basic mathematical operations as building blocks. We demonstrate this by introducing a novel framework that significantly reduces human bias through a generic search space. We believe these preliminary successes in discovering machine learning algorithms from scratch indicate a promising new direction in the field.
  arXiv  Detail & Related papers  (2020-03-06T19:00:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.