RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgets

• URL: http://arxiv.org/abs/2604.04347v1
• Date: Mon, 06 Apr 2026 01:39:13 GMT
• Title: RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgets
• Abstract summary: We present the first systematic comparison of three optimization paradigms -- Elo tournament selection, GEPA, and greedy hill-climbing.<n>RoboPhD introduces validation-free evolution: instead of splitting the budget between training and validation, it uses Elo competition on training data to simultaneously evaluate agents and drive evolution.<n>We release RoboPhD as a versatile toolkit under the MIT license with a simple optimize_anything() API for evolving diverse complex agents.
• Score: 0.2126018197183873
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 2026 has brought an explosion of interest in LLM-guided evolution of agentic artifacts, with systems like GEPA and Autoresearch demonstrating that LLMs can iteratively improve prompts, code, and agent architectures across diverse domains. As adoption accelerates, a central question emerges: given the same information, the same seed agent, and the same objective, which optimization algorithm yields the best results under the same evaluation budget? This question becomes critical when evaluations are expensive, such as when they require human judgment or multiple LLM calls. We present the first systematic comparison of three optimization paradigms -- Elo tournament selection (RoboPhD), Pareto-based selection (GEPA), and greedy hill-climbing (Autoresearch) -- across four benchmarks spanning abstract reasoning, cloud scheduling, SQL generation, and financial QA, all under a fixed budget of 1,500 evaluations. RoboPhD introduces validation-free evolution: instead of splitting the budget between training and validation, it uses Elo competition on training data to simultaneously evaluate agents and drive evolution. All three systems receive seed agents with diagnostic print() statements that evolution can grow, enabling self-instrumenting agents that develop increasingly informative diagnostics for the benefit of their evolutionary successors. Using a single default configuration, RoboPhD outperforms both GEPA and Autoresearch on three of four benchmarks, losing only on the simplest task, where the winning solution (from our Autoresearch adaptation) required under 90 lines of code. On ARC-AGI, RoboPhD evolves a 22-line seed agent into a 1,013-line multi-strategy system, improving accuracy from 27.8% to 65.8% using Gemini 3.1 Flash Lite as the solver. We release RoboPhD as a versatile toolkit under the MIT license with a simple optimize_anything() API for evolving diverse complex agents.

Related papers

• AgentFold: Closed-Loop Agentic Search for Protein Folding Model Design [27.154390712714264]
  We present AgentFold, a framework that formulates folding-model development as a closed-loop search over executable code variants.<n>Under a matched computational budget, AgentFold improves the best lDDT by 7.5% over independent Codex proposals.<n>The resulting intervention traces reveal recurring empirical design patterns.
  arXiv  Detail & Related papers  (2026-08-27T07:38:16Z)
• AI Research Preference Models [87.14303218008381]
  Research agents can propose, implement, and evaluate their own machine learning experiments.<n> progress on frontier tasks is throttled by cost.<n>We introduce AI Research Preference Models ( RPMs) that predict which of multiple candidate solutions are most worth executing.
  arXiv  Detail & Related papers  (2026-08-14T04:20:37Z)
• APEX: Automated Prompt Engineering eXpert with Dynamic Data Selection [60.504476571531]
  Large Language Models are highly sensitive to prompt formulation, necessitating automatic prompt optimization to unlock their full potential.<n>Current methods treat the development dataset as a static benchmark, wasting significant compute budget on uninformative data.<n>We introduce APEX (Automatic Prompt Engineering eXpert), a novel framework that optimize the data usage alongside the prompt search.
  arXiv  Detail & Related papers  (2026-06-09T21:22:06Z)
• InferenceBench: A Benchmark for Open-Ended LLM Inference Optimization by AI Agents [15.892896819368751]
  We introduce InferenceBench, where an agent must deploy an OpenAI-compatible inference server and optimize the speed of LLM inference.<n>Three optimization scenarios isolate distinct bottlenecks of inference.<n>Agents reliably improve over a naive PyTorch baseline.<n>Overall, InferenceBench reflects the ability of agents to operate in an open-ended AI engineering setting.
  arXiv  Detail & Related papers  (2026-05-20T15:55:21Z)
• optimize_anything: A Universal API for Optimizing any Text Parameter [98.42497715725356]
  We show a single AI-based optimization system-supporting single-task search, multi-task search with cross-problem transfer, and generalization to unseen inputs.<n>We show for the first time that text optimization with LLM-based search is a general-purpose problem-solving paradigm.
  arXiv  Detail & Related papers  (2026-05-19T10:18:12Z)
• AlphaLab: Autonomous Multi-Agent Research Across Optimization Domains with Frontier LLMs [14.167136550376497]
  We present AlphaLab, an autonomous research harness that automates the full experimental cycle in computation-intensive domains.<n>It adapts to the domain and explores the data, writing analysis code and producing a research report.<n>It runs large-scale experiments via a Strategist/Worker loop, accumulating domain knowledge in a persistent playbook.
  arXiv  Detail & Related papers  (2026-03-31T21:16:20Z)
• Dr. Zero: Self-Evolving Search Agents without Training Data [34.91191770652202]
  We introduce Dr. Zero, a framework enabling search agents to effectively self-evolve without any training data.<n>In particular, we design a self-evolution feedback loop where a proposer generates diverse questions to train a solver from the same base model.<n>To enhance training efficiency, we also introduce hop-grouped relative policy optimization (HRPO)
  arXiv  Detail & Related papers  (2026-01-11T20:27:55Z)
• RoboPhD: Self-Improving Text-to-SQL Through Autonomous Agent Evolution [0.2389431394996565]
  RoboPhD is a system where AI agents autonomously conduct research to improve Text-to-adaptive performance.<n>Central to the framework is an ELO-based selection mechanism enabling survival-of-the-fittest dynamics.<n>The system achieves 73.67% accuracy on the BIRD test set, demonstrating that AI can autonomously build a strong agentic system with only a trivial human-provided starting point.
  arXiv  Detail & Related papers  (2026-01-03T09:16:07Z)
• ROAD: Reflective Optimization via Automated Debugging for Zero-Shot Agent Alignment [1.6968020497268546]
  ROAD is a novel framework that treats optimization as a dynamic debug investigation rather than a search.<n>Road is highly sample-efficient, achieving a 5.6 percent increase in success rate and a 3.8 percent increase in search accuracy.<n>These findings suggest that mimicking the human engineering loop of failure analysis and patching offers a viable, data-efficient alternative to resource-intensive training.
  arXiv  Detail & Related papers  (2025-12-30T07:31:34Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• R&D-Agent: An LLM-Agent Framework Towards Autonomous Data Science [70.1638335489284]
  High-level machine learning engineering tasks remain labor-intensive and iterative.<n>We introduce R&D-Agent, a comprehensive, decoupled, and framework that formalizes the machine learning process.<n>R&D-Agent defines the MLE into two phases and six components, turning agent design for MLE into a principled, testable process.
  arXiv  Detail & Related papers  (2025-05-20T06:07:00Z)
• OptimAI: Optimization from Natural Language Using LLM-Powered AI Agents [8.441638148384389]
  We introduce OptimAI, a framework for solving Optimization problems described in natural language.<n>Our framework is built upon the following key roles: formulator, planner, coder and code critic.<n>Our approach attains 88.1% accuracy on the NLP4LP dataset and 82.3% on the Optibench dataset, reducing error rates by 58% and 52%, respectively, over prior best results.
  arXiv  Detail & Related papers  (2025-04-23T17:45:05Z)
• RAG-Gym: Systematic Optimization of Language Agents for Retrieval-Augmented Generation [43.50113345998687]
  We introduce RAG-Gym, a comprehensive platform that explores three optimization dimensions: (1) prompt engineering, (2) actor tuning, and (3) critic training.<n>For prompt engineering, we propose Re$2$Search, a novel agent incorporating reflection reasoning that significantly outperforms standard prompts.<n>In actor tuning, we evaluate three popular post-training algorithms with fine-grained process supervision and identify direct preference optimization as the most effective.
  arXiv  Detail & Related papers  (2025-02-19T18:56:03Z)
• DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning [56.887047551101574]
  We present DS-Agent, a novel framework that harnesses large language models (LLMs) agent and case-based reasoning (CBR) In the development stage, DS-Agent follows the CBR framework to structure an automatic iteration pipeline, which can flexibly capitalize on the expert knowledge from Kaggle. In the deployment stage, DS-Agent implements a low-resource deployment stage with a simplified CBR paradigm, significantly reducing the demand on foundational capabilities of LLMs.
  arXiv  Detail & Related papers  (2024-02-27T12:26:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.