AI Research Preference Models

• URL: http://arxiv.org/abs/2608.13940v1
• Date: Fri, 14 Aug 2026 04:20:37 GMT
• Title: AI Research Preference Models
• Abstract summary: Research agents can propose, implement, and evaluate their own machine learning experiments.<n> progress on frontier tasks is throttled by cost.<n>We introduce AI Research Preference Models ( RPMs) that predict which of multiple candidate solutions are most worth executing.
• Score: 87.14303218008381
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI research agents (AIRA) can now propose, implement, and evaluate their own machine learning experiments, but progress on frontier tasks is throttled by cost: a candidate solution can be written in minutes, whereas evaluating it can take hours to days of GPU time. An agent can therefore propose far more candidates than it can afford to run, and its progress depends on its research preference: how it allocates a fixed execution budget across many candidates. We introduce AI Research Preference Models (RPMs) that predict which of multiple candidate solutions are most worth executing, without paying the cost of executing them all. We build RPMs from frozen pretrained language models (with no task-specific training), in two forms: an inference-only model that reasons over candidate plans, code, and prior executed solutions, and an agentic model that additionally runs small-scale pilot experiments before deciding. We integrate both into the AIRA-dojo search agent and evaluate on AIRS-Bench, a recent benchmark of machine learning research tasks for AI research agents. The two variants raise the average normalized score from 0.684 to 0.711 and 0.729 respectively, and reach the unguided agent's 24-hour performance in roughly 15 hours, using less than two-thirds of its execution budget. Our best RPMs also yield new state-of-the-art results on two AIRS-Bench tasks.

Related papers

• $τ_0$-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation [51.05516457377339]
  Long-horizon robot manipulation requires a robot to both execute individual skills reliably and sequence them coherently over extended tasks.<n>We introduce $_0$-VLA, a hierarchical robot foundation model that formulates high-level subtask generation as a compute-scalable inference problem.<n>A low-level policy then executes the generated subtask across multiple robot embodiments.
  arXiv  Detail & Related papers  (2026-08-17T17:59:11Z)
• AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agents [49.67355440164857]
  We introduce AIRS-Bench, a suite of 20 tasks sourced from state-of-the-art machine learning papers.<n>Airs-Bench tasks assess agentic capabilities over the full research lifecycle.<n>We open-source the AIRS-Bench task definitions and evaluation code to catalyze further development in autonomous scientific research.
  arXiv  Detail & Related papers  (2026-02-06T16:45:02Z)
• AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent [57.10083973844841]
  AgentArk is a novel framework to distill multi-agent dynamics into the weights of a single model.<n>We investigate three hierarchical distillation strategies across various models, tasks, scaling, and scenarios.<n>By shifting the burden of computation from inference to training, the distilled models preserve the efficiency of one agent while exhibiting strong reasoning and self-correction performance of multiple agents.
  arXiv  Detail & Related papers  (2026-02-03T19:18:28Z)
• Towards Execution-Grounded Automated AI Research [106.90422658528819]
  Execution grounding may help, but it is unclear whether automated execution is feasible and whether LLMs can learn from the execution feedback.<n>We build an automated executor to implement ideas and launch large-scale parallel GPU experiments to verify their effectiveness.<n>We analyze two methods to learn from the execution feedback: evolutionary search and reinforcement learning.
  arXiv  Detail & Related papers  (2026-01-20T22:35:44Z)
• A Rosetta Stone for AI Benchmarks [28.690200241767897]
  Most AI benchmarks saturate within years or even months after they are introduced, making it hard to study long-run trends in AI capabilities.<n>We build a statistical framework that stitches benchmarks together, putting model capabilities and benchmark difficulties on a single numerical scale.<n>This acts as a "Rosetta Stone", allowing us to compare models across a wide range of abilities and time, even if they are not evaluated on the same benchmarks.
  arXiv  Detail & Related papers  (2025-11-28T20:18:58Z)
• AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress [71.02263260394261]
  Large language models (LLMs) still encounter challenges in multi-turn decision-making tasks.<n>We build process reward models (PRMs) to evaluate each decision and guide the agent's decision-making process.<n>AgentPRM captures both the interdependence between sequential decisions and their contribution to the final goal.
  arXiv  Detail & Related papers  (2025-11-11T14:57:54Z)
• RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts [4.06186944042499]
  We introduce RE-Bench, which consists of 7 challenging, open-ended ML research engineering environments and data from 71 8-hour attempts by 61 human experts.<n>We find that the best AI agents achieve a score 4x higher than human experts when both are given a total time budget of 2 hours per environment.<n>Humans currently display better returns to increasing time budgets, narrowly exceeding the top AI agent scores given an 8-hour budget, and achieving 2x the score of the top AI agent when both are given 32 total hours (across different attempts).
  arXiv  Detail & Related papers  (2024-11-22T18:30:46Z)
• MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation [96.71370747681078]
  We introduce MLAgentBench, a suite of 13 tasks ranging from improving model performance on CIFAR-10 to recent research problems like BabyLM. For each task, an agent can perform actions like reading/writing files, executing code, and inspecting outputs. We benchmark agents based on Claude v1.0, Claude v2.1, Claude v3 Opus, GPT-4, GPT-4-turbo, Gemini-Pro, and Mixtral and find that a Claude v3 Opus agent is the best in terms of success rate.
  arXiv  Detail & Related papers  (2023-10-05T04:06:12Z)
• AIBench Training: Balanced Industry-Standard AI Training Benchmarking [26.820244556465333]
  Earlier-stage evaluations of a new AI architecture/system need affordable benchmarks. We use real-world benchmarks to cover the factors space that impacts the learning dynamics. We contribute by far the most comprehensive AI training benchmark suite.
  arXiv  Detail & Related papers  (2020-04-30T11:08:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.