AutoResearch: An Execution-Grounded Multi-Agent Framework for Reliable Research Workflow Automation

• URL: http://arxiv.org/abs/2607.02520v1
• Date: Mon, 04 May 2026 08:24:21 GMT
• Title: AutoResearch: An Execution-Grounded Multi-Agent Framework for Reliable Research Workflow Automation
• Abstract summary: AutoResearch is an execution-grounded multi-agent framework for reliable research workflow automation.<n>System treats errors, citation-verification failures, and review-agent feedback as practical filtering signals for generated research artifacts.
• Score: 8.807417226975081
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated research agents increasingly generate code, retrieve literature, and draft scientific artifacts, but they often fail to verify whether generated experiments execute correctly or whether cited sources support generated claims. We present AutoResearch, an execution-grounded multi-agent framework for reliable research workflow automation. AutoResearch couples sandboxed Python/PyTorch execution, iterative code repair, citation verification, claim-support auditing, decision control, and structured \LaTeX{} artifact generation. The system treats runtime errors, citation-verification failures, and review-agent feedback as practical filtering signals for generated research artifacts. In controlled evaluations on HumanEval, MBPP, a SciCode subset, citation-validation tasks, claim-support auditing, and small end-to-end workflow stress tests, AutoResearch improves execution success, citation validity, local claim support, and workflow completion relative to directly comparable baselines. Code-oriented agents are reported separately as partial comparisons. AutoResearch is intended as a reliability-oriented research assistant, not as a fully autonomous scientist or a standalone manuscript-quality benchmark. Source Code: https://github.com/raja21068/AutoResearch

Related papers

• ToFu: A White-Box, Token-Efficient Agent Harness for Researchers [73.72929185946842]
  ToFu is an agentic harness for researchers that reads your files, edits files, runs commands, and integrates with your development tools.<n>As a research assistant, it supports practical research with superior token efficiency, lower cost, and multilingual capability.<n>As a research object, ToFu provides a white-box agentic harness that allows researchers to inspect, modify, and evaluate its orchestration logic, tool-use behavior, and harness design.
  arXiv  Detail & Related papers  (2026-07-13T11:26:06Z)
• VERITAS: Towards a General-Purpose Replication Tool for Scientific Research [19.45546798679836]
  We present VERITAS, a domain-agnostic replication framework built around CLI coding agents.<n>Given a paper, a code repository, or both, VERITAS extracts the paper's claims, runs the methodology while resolving issues as they arise, and judges each claim against the evidence from experiment runs.<n>The pipeline returns an importance-weighted Replication Score, a severity-rated log of every fix applied, and the patched.<n>We evaluate VERITAS on CORE-Bench and ReplicationBench, 65 papers spanning computer science, social science, medicine, and astrophysics.
  arXiv  Detail & Related papers  (2026-07-03T03:55:59Z)
• Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories [51.22051230894794]
  Evaluation based on final answers shows whether an agent succeeds, but not which parts of the trajectory make the answer unreliable.<n>We collect 2,790 real trajectories from two agent frameworks, three backbone models, and three benchmarks, convert raw logs into semantic spans, and annotate harmful error spans through expert review.<n>We propose DRIFT, a claim-centric auditing framework that tracks agent claims, checks their support in trajectory evidence, and marks spans where unsupported or conflicting claims affect the answer path.
  arXiv  Detail & Related papers  (2026-06-01T10:50:26Z)
• Sibyl-AutoResearch: Autonomous Research Needs Self-Evolving Trial-and-Error Harnesses, Not Paper Generators [37.075000666622074]
  We introduce Sibyl-AutoResearch, a self-evolving AutoResearch framework built around Scientific Trial-and-Error Harnesses.<n>A harness lets agents run bounded trials, preserve positive and negative outcomes, and route lessons into later planning, validation, claim scope, scheduling, critique, writing, and repair.<n>We implement the framework in SIBYL, a file-backed autonomous research system that exposes the state, roles, memory, gates, and artifact traces needed to inspect these conversion paths.
  arXiv  Detail & Related papers  (2026-05-21T11:29:08Z)
• AI for Auto-Research: Roadmap & User Guide [107.0834449839233]
  We present an end-to-end analysis of AI across the complete research lifecycle.<n>We identify a sharp, stage-dependent boundary between reliable assistance and unreliable autonomy.<n>We show that greater automation can obscure rather than eliminate failure modes.
  arXiv  Detail & Related papers  (2026-05-18T17:08:26Z)
• Can Coding Agents Reproduce Findings in Computational Materials Science? [49.254975563645786]
  We present AutoMat, a benchmark for evaluating large language models' ability to reproduce scientific claims.<n>By working closely with subject matter experts, we curate a set of claims from real materials science papers to test whether coding agents can recover and execute the end-to-end workflow.<n>Results show that current LLM-based agents obtain low overall success rates on AutoMat, with the best-performing setting achieving a success rate of only 54.1%.
  arXiv  Detail & Related papers  (2026-05-01T17:42:12Z)
• RESCORE: LLM-Driven Simulation Recovery in Control Systems Research Papers [18.24843021913362]
  Reconstructing numerical simulations from control systems research papers is often hindered by underspecified parameters and ambiguous implementation details.<n>We define the task of Paper to Simulation Recoverability, the ability of an automated system to generate executable code that faithfully reproduces a paper's results.<n>We propose RESCORE, a three component LLM agentic framework, Analyzer, Coder, and Verifier.<n>Our method successfully recovers task coherent simulations for 40.7% of benchmark instances, outperforming single pass generation.
  arXiv  Detail & Related papers  (2026-04-06T00:13:14Z)
• Automating Computational Reproducibility in Social Science: Comparing Prompt-Based and Agent-Based Approaches [0.3637090866781549]
  Reproducing computational research is often assumed to be as simple as rerunning the original code with provided data.<n>This study investigates whether large language models and AI agents can automate the diagnosis and repair of such failures.
  arXiv  Detail & Related papers  (2026-02-09T11:59:59Z)
• From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking [48.90371827091671]
  AutoExperiment is a benchmark that evaluates AI agents' ability to implement and run machine learning experiments.<n>We evaluate state-of-the-art agents and find that performance degrades rapidly as $n$ increases.<n>Our findings highlight critical challenges in long-horizon code generation, context retrieval, and autonomous experiment execution.
  arXiv  Detail & Related papers  (2025-06-24T15:39:20Z)
• AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage [62.049868205196425]
  AutoReproduce is a framework capable of automatically reproducing experiments described in research papers in an end-to-end manner.<n>Results show that AutoReproduce achieves an average performance gap of $22.1%$ on $89.74%$ of the executable experiment runs.
  arXiv  Detail & Related papers  (2025-05-27T03:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.