Recovering Wasted Compute in Autoresearch Agents

• URL: http://arxiv.org/abs/2608.10424v1
• Date: Tue, 11 Aug 2026 03:15:08 GMT
• Title: Recovering Wasted Compute in Autoresearch Agents
• Abstract summary: We study the modeling pipeline at the core of autoresearch systems and identify common failure modes.<n>We find that large gains in autoresearch agent performance are achievable through agentic design alone.
• Score: 35.718169903054736
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A slew of recent works develop agents for solving research problems end-to-end, a paradigm increasingly referred to as autoresearch. Such agents have inspired large industry investment, motivated by their potential to automate time-consuming human labor and customize machine learning solutions for specialized applications. In this paper, we study the modeling pipeline at the core of these autoresearch systems and identify common failure modes when they are applied to tabular datasets: (1) they waste compute resolving the same bugs over and over again; (2) they often fail to tune hyperparameters even when they have a large remaining compute budget; (3) the tree-search algorithms that power them do not explore; and (4) they perform data analysis, mimicking the humans whose data they are trained on, but do not use that analysis to make downstream decisions. We explore targeted interventions and find that a global debug consultant that shares discovered runtime constraints across all branches of the search tree, prompt- and control-level enhancements, and refined tree-search algorithms successfully recover wasted compute. Our results show that large gains in autoresearch agent performance are achievable through agentic design alone, holding the underlying language model fixed.

Related papers

• How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks [6.001213285107465]
  AutoResearchEval features 100 tasks grounded in published frontier science across 7 scientific domains.<n> evaluating 8 harness-model combinations yields 800 autoresearch agent trajectories.<n>We leverage a human-calibrated agent-as-a-judge pipeline to inspect complete trajectories and intermediate artifacts.
  arXiv  Detail & Related papers  (2026-08-14T21:39:38Z)
• The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows [72.9051182568928]
  We present ReASearch, a unified framework for reasoning-driven optimization.<n>With a shared agent loop and domain-specific tools, ReASearch instantiates the exact same scaffold to optimize prompts, programs, and ML.<n>We observe that complex search behaviors emerge naturally from the agent's reasoning process.
  arXiv  Detail & Related papers  (2026-08-07T02:15:39Z)
• A case study of evaluating AI agents on a neuroscience data-to-discovery pipeline [4.751533658796929]
  We present an empirical study of general-purpose coding agents on a fly optogenetics data-to-discovery pipeline.<n>We assess agents on tasks substantially larger than existing benchmarks, datasets orders of magnitude bigger, and evaluation criteria grounded in domain expert standards.<n>We show that agents can solve several individual pipeline stages, suggesting stage-level automation is tractable.
  arXiv  Detail & Related papers  (2026-06-05T15:38:18Z)
• AI-Supervisor: Autonomous AI Research Supervision via a Persistent Research World Model [1.14219428942199]
  Existing automated research systems operate as stateless, linear pipelines.<n>We present textbfAI-Supervisor, a multi-agent orchestration framework.<n>Agents provide end-to-end AI research supervision driven by human interests.
  arXiv  Detail & Related papers  (2026-03-25T15:16:51Z)
• Cost-Aware Diffusion Active Search [8.238352995483533]
  Active search for recovering objects of interest requires trading off exploration of unknown environments with exploitation of prior observations in the search space.<n>In this work, we leverage the sequence modeling abilities of diffusion models to sample lookahead action sequences that balance the exploration-exploitation trade-off for active search without building an exhaustive search tree.<n>Our proposed algorithm outperforms standard baselines in offline reinforcement learning in terms of full recovery rate and is computationally more efficient than tree search in cost-aware active decision making.
  arXiv  Detail & Related papers  (2026-02-23T06:11:51Z)
• SelfAI: Building a Self-Training AI System with LLM Agents [79.10991818561907]
  SelfAI is a general multi-agent platform that combines a User Agent for translating high-level research objectives into standardized experimental configurations.<n>An Experiment Manager orchestrates parallel, fault-tolerant training across heterogeneous hardware while maintaining a structured knowledge base for continuous feedback.<n>Across regression, computer vision, scientific computing, medical imaging, and drug discovery benchmarks, SelfAI consistently achieves strong performance and reduces redundant trials.
  arXiv  Detail & Related papers  (2025-11-29T09:18:39Z)
• Barbarians at the Gate: How AI is Upending Systems Research [58.95406995634148]
  We argue that systems research, long focused on designing and evaluating new performance-oriented algorithms, is particularly well-suited for AI-driven solution discovery.<n>We term this approach as AI-Driven Research for Systems ( ADRS), which iteratively generates, evaluates, and refines solutions.<n>Our results highlight both the disruptive potential and the urgent need to adapt systems research practices in the age of AI.
  arXiv  Detail & Related papers  (2025-10-07T17:49:24Z)
• DatasetResearch: Benchmarking Agent Systems for Demand-Driven Dataset Discovery [26.388978716803464]
  Can AI agents transcend conventional search to systematically discover any dataset that meets specific user requirements?<n>Our benchmark and comprehensive analysis provide the foundation for the next generation of self-improving AI systems.
  arXiv  Detail & Related papers  (2025-08-09T12:15:08Z)
• From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents [96.65646344634524]
  Large Language Models (LLMs), endowed with reasoning and agentic capabilities, are ushering in a new paradigm termed Agentic Deep Research.<n>We trace the evolution from static web search to interactive, agent-based systems that plan, explore, and learn.<n>We demonstrate that Agentic Deep Research not only significantly outperforms existing approaches, but is also poised to become the dominant paradigm for future information seeking.
  arXiv  Detail & Related papers  (2025-06-23T17:27:19Z)
• Tree Search for Language Model Agents [73.97960454223164]
  We propose an inference-time search algorithm for LM agents to perform exploration and multi-step planning in interactive web environments.<n>Our approach is a form of best-first tree search that operates within the actual environment space.<n>It is the first tree search algorithm for LM agents that shows effectiveness on realistic web tasks.
  arXiv  Detail & Related papers  (2024-07-01T17:07:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.