CliffSearch: Structured Agentic Co-Evolution over Theory and Code for Scientific Algorithm Discovery

• URL: http://arxiv.org/abs/2604.01210v1
• Date: Wed, 01 Apr 2026 17:51:26 GMT
• Title: CliffSearch: Structured Agentic Co-Evolution over Theory and Code for Scientific Algorithm Discovery
• Abstract summary: We present CliffSearch, an agentic evolutionary framework in which the core evolution operators are implemented as LLM agents.<n>We illustrate the framework on three benchmark-grounded studies: transformer hyper-connection evolution, discovery on a fixed nanoGPT stack, and a smaller native-optimizer ablation.
• Score: 8.600891931062277
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific algorithm discovery is iterative: hypotheses are proposed, implemented, stress-tested, and revised. Current LLM-guided search systems accelerate proposal generation, but often under-represent scientific structure by optimizing code-only artifacts with weak correctness/originality gating. We present CliffSearch, an agentic evolutionary framework in which the core evolution operators (pair selection, crossover, mutation, and review) are implemented as LLM agents, and the loop is designed around three principles: (1) each node is a structured scientific artifact, instantiated in either theory+code or code_only mode, (2) reviewer judgments of correctness and originality are first-class selection gates alongside optimization of the benchmark metric of interest, and (3) mutation is split into exploration and correction pathways with distinct objectives. Exploration mutation imports ideas from adjacent scientific domains to increase novelty, while correction mutation performs targeted evidence-guided repair using reviewer signals over theory, code, benchmark results, and runtime errors. We illustrate the framework on three benchmark-grounded studies: transformer hyper-connection evolution, optimizer discovery on a fixed nanoGPT stack, and a smaller native-optimizer ablation. Across these settings, the same loop supports explicit metric direction, reproducible persistence, and reviewer-gated comparison of discoveries under controlled search conditions. The result is a discovery workflow that prioritizes scientific interpretability and correctness while optimizing task metrics under controlled novelty constraints, rather than maximizing candidate throughput alone. Full run artifacts, interactive visualizations, and exported best nodes for the reported studies are available at https://cliffsearch.ai .

Related papers

• GAE: Graph-Augmented Evolution for Scientific Discovery via Reinforcement Optimization [1.7188280334580195]
  We present GAE (Graph-Augmented Evolution), a framework for program search guided by Large Language Models (LLMs)<n>We use a relational graph neural network (GNN) to parse programs into typed computation graphs, producing structure-aware embeddings.<n>An RL-optimized meta-controller leverages these embeddings to replace blind evolutionary sampling with a directed policy.
  arXiv  Detail & Related papers  (2026-07-11T05:32:54Z)
• OPTScientist: Multi-Agent Discovery of Typed Optimizer Programs for Transformer Pretraining [22.097466841331258]
  We introduce OPTScientist, a theory-guided multi-agent framework for discovery in a typed domain-specific language.<n>Four role agents, Theorist, Designer, Engineer, and Reviewer, collaborate within a single orchestration loop to propose, hypotheses, compile, evaluate, and critique results.<n>Our results suggest a path toward automated science grounded in theory, typed programs, compiler validation, and closed-loop experimentation.
  arXiv  Detail & Related papers  (2026-06-02T15:21:53Z)
• From I/O to Code with Discovery Agent [103.88427301265669]
  We propose DIO-Agent, a discovery agent for IO2Code.<n>Our method frames IO2Code as an evolutionary search over discrete program space.<n>Extensive experiments show that DIO-Agent consistently outperforms both traditional program-by-example method and SOTA evolution-agent baselines.
  arXiv  Detail & Related papers  (2026-05-14T18:57:32Z)
• Evaluation-driven Scaling for Scientific Discovery [77.20820317940581]
  We introduce Simple Test-time Evaluation-driven Scaling (SimpleTES), a framework that combines parallel exploration, feedback-driven refinement, and local selection.<n>We find substantial gains unlocked by scaling evaluation-driven discovery loops along the right dimensions.<n>Our results establish effective evaluation-driven loop scaling as a central axis for advancing scientific discovery.
  arXiv  Detail & Related papers  (2026-04-21T11:24:09Z)
• AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation [59.964989458924585]
  AdaExplore is an agent framework that enables self-improvement via accumulated execution feedback for performance-critical kernel code generation.<n>In the adaptation stage, the agent synthesizes tasks and converts recurring failures into a reusable memory of validity rules.<n>In the search stage, the agent organizes candidate kernels as a tree and alternates between small local refinements and larger structural regeneration.
  arXiv  Detail & Related papers  (2026-04-17T18:25:03Z)
• OR-Agent: Bridging Evolutionary Search and Structured Research for Automated Algorithm Discovery [10.217363774023033]
  OR-Agent is a multi-agent research framework designed for automated exploration in rich experimental environments.<n>We introduce an evolutionary-systematic mechanism that unifies evolutionary selection of research starting points, comprehensive research plan generation, and coordinated exploration within a research tree.<n>We conduct experiments across classical optimization benchmarks-including traveling salesman, capacitated vehicle routing, bin packing, orienteering, and multiple knapsack problems-as well as a simulation-based cooperative driving scenarios.
  arXiv  Detail & Related papers  (2026-02-14T13:32:03Z)
• Continuous Program Search [4.198653054660764]
  We frame this as an operator-design problem: learn a continuous program space where latent distance has behavioral meaning, then design mutation operators that exploit this structure without changing the evolutionary algorithm.<n>We make locality measurable by tracking action-level divergence under controlled latent perturbations, identifying an empirical trust region for behavior-local continuous variation.<n>Although isotropic mutation occasionally attains higher peak performance, geometry-compiled mutation yields faster, more reliable progress, demonstrating that semantically aligned mutation can substantially improve search efficiency without modifying the underlying evolutionary algorithm.
  arXiv  Detail & Related papers  (2026-02-07T18:41:14Z)
• Efficient Differentiable Causal Discovery via Reliable Super-Structure Learning [51.20606796019663]
  We propose ALVGL, a novel and general enhancement to the differentiable causal discovery pipeline.<n>ALVGL employs a sparse and low-rank decomposition to learn the precision matrix of the data.<n>We show that ALVGL not only achieves state-of-the-art accuracy but also significantly improves optimization efficiency.
  arXiv  Detail & Related papers  (2026-01-09T02:18:59Z)
• CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning [63.44477226386808]
  Chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks.<n>But it struggles in scientific domains where even advanced models often produce incorrect or superficial reasoning.<n>We propose CoT-Evo, an evolutionary CoT distillation framework to overcome this problem.
  arXiv  Detail & Related papers  (2025-10-15T05:29:56Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Beyond Progress Measures: Theoretical Insights into the Mechanism of Grokking [50.465604300990904]
  Grokking refers to the abrupt improvement in test accuracy after extended overfitting.<n>In this work, we investigate the grokking mechanism underlying the Transformer in the task of prime number operations.
  arXiv  Detail & Related papers  (2025-04-04T04:42:38Z)
• Regulatory DNA sequence Design with Reinforcement Learning [56.20290878358356]
  We propose a generative approach that leverages reinforcement learning to fine-tune a pre-trained autoregressive model.<n>We evaluate our method on promoter design tasks in two yeast media conditions and enhancer design tasks for three human cell types.
  arXiv  Detail & Related papers  (2025-03-11T02:33:33Z)
• An information-theoretic evolutionary algorithm [0.0]
  It-EA iteratively updates a search distribution with two parameters, the center, that is the bit vector at which standard bit mutation is applied, and the mutation rate. In an empirical analysis, on OneMax and LeadingOnes, the elitist and non elitist it-EAs obtain promising results.
  arXiv  Detail & Related papers  (2023-04-12T16:36:22Z)
• Self-Adjusting Evolutionary Algorithms for Multimodal Optimization [0.0]
  Self-adjusting and self-adaptive mechanisms can provably outperform static settings in evolutionary algorithms for binary search spaces. We suggest a mechanism called stagnation detection that can be added as a module to existing evolutionary algorithms.
  arXiv  Detail & Related papers  (2020-04-07T11:02:10Z)
• On the Performance of Metaheuristics: A Different Perspective [0.0]
  We study some basic evolutionary and swam-intelligence metaheuristics i.e. Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC), Teaching-Learning-Based Optimization (TLBO) and Cuckoo Optimization algorithm (COA) A large number of experiments have been conducted on 20 different optimization benchmark functions with different characteristics, and the results reveal to us some fundamental conclusions besides the following ranking order among these metaheuristics.
  arXiv  Detail & Related papers  (2020-01-24T09:34:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.