Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents

• URL: http://arxiv.org/abs/2607.09195v1
• Date: Fri, 10 Jul 2026 08:39:30 GMT
• Title: Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents
• Abstract summary: Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery.<n>Here we propose the Hypothesis Evolution Protocol (HEP), an agent harness that provides hypothesis generation, evaluation, and evolution as explicit, auditable operations.<n>On materials-science research tasks, a HEP-equipped agent operates the hypothesis--test--evidence--belief cycle that planning-style agents lack, generalizes across research questions, and exploits the protocol more fully as the base LLM becomes more capable.
• Score: 0.42970700836450476
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery. Equipped with broad knowledge, flexible reasoning, and tool use, they have the potential to autonomously explore and solve scientific problems by repeatedly proposing hypotheses, testing them, and revising their beliefs in the light of the evidence. In current agents, however, these hypotheses, tests, and belief updates are buried in unstructured logs, and no mechanism lets the agent or the human researcher audit that process. Here we propose the Hypothesis Evolution Protocol (HEP), an agent harness that provides hypothesis generation, evaluation, and evolution as explicit, auditable operations. On materials-science research tasks, a HEP-equipped agent operates the hypothesis--test--evidence--belief cycle that planning-style agents lack, generalizes across research questions, and exploits the protocol more fully as the base LLM becomes more capable. These results mark a step toward auditable AI scientists, whose scientific reasoning can be inspected, verified, and built upon.

Related papers

• LabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI Agents [54.20891951945591]
  Methods developed by predecessors are often expanded upon by new researchers to explore more novel and in-depth scientific questions.<n>The change of lab staff, such as student graduation, leads to a lack of personnel capable of replicating methods.<n>We propose LabAgent, a reproduce and discovery harness tailored for a lab's continuous work.
  arXiv  Detail & Related papers  (2026-09-11T18:53:18Z)
• EvoSCM: Scientific Belief Revision Through Causal Model Evolution and Experimentation [68.80901568810616]
  EvoSCM aims to equip scientific agents with explicit structural causal models that evolve as new evidence is collected.<n>We evaluate EvoSCM onPhysics, a benchmark requiring agents to uncover the hidden dynamics of noncanonical physical worlds through experimentation.
  arXiv  Detail & Related papers  (2026-09-01T16:55:56Z)
• AI Scientists as Engines of Discovery: A Case for Development within Reformed Institutions [0.10102237605686408]
  Agentic artificial intelligence (AI) systems are beginning to assist, accelerate, and partially automate scientific discovery.<n>We argue that suitably designed multi-agent systems may evolve from passive computational tools into AI scientists''<n>Such systems must be developed and deployed within a scientific ecosystem fit for purpose.
  arXiv  Detail & Related papers  (2026-06-22T05:06:29Z)
• The Age of AI Agents Demands A New Scientific Paradigm To Sustain Trustworthy Science [0.0]
  We argue that science must evolve its verification infrastructure, as it has before with peer review.<n>We propose criteria for an adapted verification infrastructure that emphasizes observable-by-default verification, scalable results, and clear attribution.
  arXiv  Detail & Related papers  (2026-06-18T20:38:50Z)
• Benchmarking AI Agents for Addressing Scientific Challenges Across Scales [118.2204632627895]
  SciAgentArena is a systematic benchmark for evaluating AI agents in real-world scientific research scenarios.<n>It comprises approximately 200 tasks with stepwise verification and an interactive, agent-agnostic environment for assessing diverse AI agents.<n>We find that current agents can contribute effectively to well-specified data-analysis, particularly when the task structure and evaluation criteria are clear.
  arXiv  Detail & Related papers  (2026-06-10T22:55:30Z)
• AI scientists produce results without reasoning scientifically [3.100302590436282]
  Large language model (LLM)-based systems are increasingly deployed to conduct scientific research autonomously.<n>Here, we evaluate LLM-based scientific agents across eight domains, spanning execution to hypothesis-driven inquiry.
  arXiv  Detail & Related papers  (2026-04-20T20:23:42Z)
• FIRE-Bench: Evaluating Agents on the Rediscovery of Scientific Insights [63.32178443510396]
  We introduce FIRE-Bench (Full-cycle Insight Rediscovery Evaluation), a benchmark that evaluates agents through the rediscovery of established findings.<n>Even the strongest agents achieve limited rediscovery success (50 F1), exhibit high variance across runs, and display recurring failure modes in experimental design, execution, and evidence-based reasoning.
  arXiv  Detail & Related papers  (2026-02-02T23:21:13Z)
• AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite [75.58737079136942]
  We present AstaBench, a suite that provides the first holistic measure of agentic ability to perform scientific research.<n>Our suite comes with the first scientific research environment with production-grade search tools.<n>Our evaluation of 57 agents across 22 agent classes reveals several interesting findings.
  arXiv  Detail & Related papers  (2025-10-24T17:10:26Z)
• Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physics [82.55776608452017]
  Large language models (LLMs) provide a flexible and versatile framework that orchestrates interactions with human scientists, natural language, computer language and code, and physics.<n>This paper presents our view and vision of LLM-based scientific agents and their growing role in transforming the scientific discovery lifecycle.<n>We identify open research challenges and outline promising directions for building more robust, generalizable, and adaptive scientific agents.
  arXiv  Detail & Related papers  (2025-10-10T22:26:26Z)
• AIGS: Generating Science from AI-Powered Automated Falsification [17.50867181053229]
  We propose Baby-AIGS as a baby-step demonstration of a full-process AIGS system, which is a multi-agent system with agents in roles representing key research process. Experiments on three tasks preliminarily show that Baby-AIGS could produce meaningful scientific discoveries, though not on par with experienced human researchers.
  arXiv  Detail & Related papers  (2024-11-17T13:40:35Z)
• Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System [62.832818186789545]
  Virtual Scientists (VirSci) is a multi-agent system designed to mimic the teamwork inherent in scientific research.<n>VirSci organizes a team of agents to collaboratively generate, evaluate, and refine research ideas.<n>We show that this multi-agent approach outperforms the state-of-the-art method in producing novel scientific ideas.
  arXiv  Detail & Related papers  (2024-10-12T07:16:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.