The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators

• URL: http://arxiv.org/abs/2606.26294v2
• Date: Mon, 29 Jun 2026 17:13:25 GMT
• Title: The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
• Abstract summary: Self-improving agents are state-of-the-art (SOTA) on agentic coding benchmarks and have recently been extended to general domains.<n>We introduce the Red Queen Godel Machine (RQGM), an evolutionary framework for self-improvement under non-stationary utilities.<n>We show that the RQGM improves test pass rate over the prior SOTA by adding a complementary agent-as-a-judge code-review signal.<n>We then turn to scientific paper writing and reviewing, and Olympiad-level proof writing and grading, where the RQGM improves performance over prior self-improving agents
• Score: 17.44053709945519
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-improving agents are state-of-the-art (SOTA) on agentic coding benchmarks and have recently been extended to general domains. However, their search methods generally assume a stationary evaluation criterion: a fixed verifier, benchmark, or labeled dataset that remains valid as the agent improves. This ignores a central feature of evolution: species adapt as their environments change with them. We aim to bring the same principle to recursive self-improvement, making evaluation part of the improvement loop and opening search to evolving evaluators, adversarial objectives, and dynamic utilities that may surpass static benchmarks. We introduce the Red Queen Godel Machine (RQGM), an evolutionary framework for recursive self-improvement under non-stationary utilities. The RQGM makes this possible through controlled utility evolution: search is organized into epochs with a fixed within-epoch evaluation criterion, while the utility can be updated at epoch boundaries, so self-improvement guarantees hold per epoch as the objective evolves across them. We begin by showing that even on verifiable coding tasks, the RQGM improves test pass rate over the prior SOTA by adding a complementary agent-as-a-judge code-review signal. This signal is cheaper and the RQGM uses 1.35x-1.72x fewer tokens. We then turn to scientific paper writing and reviewing, and Olympiad-level proof writing and grading, where the RQGM improves performance over prior self-improving agents: co-evolved writers reach 1.78x-1.86x higher acceptance rates under a diverse agent-as-a-judge panel, while co-evolved graders reach 9% higher ground-truth accuracy. In paper reviewing, the strongest baseline reviewer over-accepts AI-generated papers at up to 1.91x the human rate. The RQGM corrects this by introducing an adversarial objective that discovers reviewers equally stringent on AI and human work.

Related papers

• AgentOPSD: Recursive Self-Distillation for Agentic Reinforcement Learning [58.76655851910778]
  AgentOPSD is a critic-free, recursive method for turn-level credit assignment in agentic reinforcement learning.<n>We evaluate AgentOPSD on ALFWorld, WebShop, and Search-QA using Qwen2.5 models at two scales (3B and 7B)
  arXiv  Detail & Related papers  (2026-08-06T13:00:59Z)
• SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning [14.505575259468053]
  Test-time reinforcement learning (TTRL) enables language models to self-evolve at inference time without labeled feedback.<n>We introduce SERPO, which replaces answer voting with a closed loop that co-evolves response evidence, querytokens, and policy parameters.<n>New actor rollouts then refresh both the archives and rubrics, closing the three-way evolution loop.
  arXiv  Detail & Related papers  (2026-07-29T13:03:07Z)
• Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation [37.16322609017877]
  Messier is a unified corpus of 957,253 records that span 30 benchmarks, 714 agents, 11,891 tasks, and 74,205 verifiers.<n>M Messier consolidates public benchmark scores and supplements them with five-agent runs across six underrepresented professional and scientific domains.
  arXiv  Detail & Related papers  (2026-07-28T15:50:19Z)
• Self-Modifying Lean Proof Agents with Verifier-Grounded Benchmark Coevolution [20.44043749792232]
  We present a self-evolving Lean proof agent in which a small fixed workspace wraps a fully mutable proof workflow, prompts, and tools.<n>Unlike most self-evolving systems, which optimize against a fixed external benchmark, our system coevolves the agent and its benchmark.<n>The best coevolving agent reaches a 45.1% held-out solve rate, versus 12.7% for the seed and 32.0% for the best fixed-benchmark agent.
  arXiv  Detail & Related papers  (2026-07-19T17:26:59Z)
• The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development? [80.24951682268332]
  We introduce the Meta-Agent Challenge (MAC), an evaluation framework designed to test the capacity of frontier models for autonomous agent development.<n>To ensure evaluation integrity, this framework is secured by multi-layer defenses against reward hacking.<n>We demonstrate that meta-agents rarely match human-engineered baseline policies, and the few that do are dominated by proprietary frontier models.
  arXiv  Detail & Related papers  (2026-06-03T04:58:17Z)
• FRESCO: Benchmarking and Optimizing Re-rankers for Evolving Semantic Conflict in Retrieval-Augmented Generation [73.22935457705057]
  We introduce FRESCO, a benchmark for evaluating re-rankers in temporally dynamic contexts.<n>By pairing recency-seeking queries with historical Wikipedia revisions, FRESCO tests whether re-rankers can prioritize factually recent evidence while maintaining semantic relevance.<n>Our evaluation reveals a consistent failure mode across existing re-rankers: a strong bias toward older, semantically rich documents, even when they are factually obsolete.
  arXiv  Detail & Related papers  (2026-04-14T17:04:25Z)
• Are we still able to recognize pearls? Machine-driven peer review and the risk to creativity: An explainable RAG-XAI detection framework with markers extraction [7.723181091241251]
  We introduce an explainable framework (RAG-XAI) for assessing review quality and detecting automated patterns.<n>The framework achieves near-perfect detection performance, with XGBoost, Random Forest and LightGBM reaching 99.61% accuracy, AUC-ROC above 0.999 and F1-scores of 0.9925 on the test set.
  arXiv  Detail & Related papers  (2026-04-09T08:25:49Z)
• PaperRepro: Automated Computational Reproducibility Assessment for Social Science Papers [33.12402746591649]
  PaperRepro is a novel two-stage, multi-agent approach for automated assessment.<n>In the execution stage, agents execute the reproduction package and edit the code to capture reproduced results as explicit artifacts.<n>In the evaluation stage, agents evaluate agents using explicit evidence.
  arXiv  Detail & Related papers  (2026-02-10T09:04:59Z)
• R-Align: Enhancing Generative Reward Models through Rationale-Centric Meta-Judging [69.96389360650072]
  We show that reasoning fidelity is highly predictive of downstream RLHF outcomes, beyond standard label accuracy.<n>We propose Rationale-Centric Alignment, R-Align, which augments training with gold judgments and explicitly supervises rationale alignment.
  arXiv  Detail & Related papers  (2026-02-06T15:17:11Z)
• A2Eval: Agentic and Automated Evaluation for Embodied Brain [26.357063836707223]
  Current embodied VLM evaluation relies on static, expert-defined, manually annotated benchmarks.<n>Agentic Automatic Evaluation (A2Eval) is the first agentic framework that automates benchmark curation and evaluation through two collaborative agents.<n> Evaluated across 10 benchmarks and 13 models, A2Eval compresses evaluation suites by 85%, reduces overall computational costs by 77%, and delivers a 4.6x speedup.
  arXiv  Detail & Related papers  (2026-02-02T04:55:27Z)
• RATE: Reviewer Profiling and Annotation-free Training for Expertise Ranking in Peer Review Systems [6.083097040417168]
  We introduce LR-bench, a benchmark curated from 2024-2025 AI/NLP manuscripts with five-level self-assessed familiarity ratings.<n>We also propose RATE, a reviewer-centric ranking framework that distills each reviewer's recent publications into compact keyword-based profiles.<n>Our approach consistently achieves state-of-the-art performance, outperforming strong embedding baselines by a clear margin.
  arXiv  Detail & Related papers  (2026-01-27T14:13:46Z)
• Alita-G: Self-Evolving Generative Agent for Agent Generation [54.49365835457433]
  We present ALITA-G, a framework that transforms a general-purpose agent into a domain expert.<n>In this framework, a generalist agent executes a curated suite of target-domain tasks.<n>It attains strong gains while reducing computation costs.
  arXiv  Detail & Related papers  (2025-10-27T17:59:14Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)
• RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning [64.46921169261852]
  RAG-Zeval is a novel end-to-end framework that formulates faithfulness and correctness evaluation as a rule-guided reasoning task.<n>Our approach trains evaluators with reinforcement learning, facilitating compact models to generate comprehensive and sound assessments.<n>Experiments demonstrate RAG-Zeval's superior performance, achieving the strongest correlation with human judgments.
  arXiv  Detail & Related papers  (2025-05-28T14:55:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.