AI scientists produce results without reasoning scientifically

• URL: http://arxiv.org/abs/2604.18805v1
• Date: Mon, 20 Apr 2026 20:23:42 GMT
• Title: AI scientists produce results without reasoning scientifically
• Abstract summary: Large language model (LLM)-based systems are increasingly deployed to conduct scientific research autonomously.<n>Here, we evaluate LLM-based scientific agents across eight domains, spanning execution to hypothesis-driven inquiry.
• Score: 3.100302590436282
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM)-based systems are increasingly deployed to conduct scientific research autonomously, yet whether their reasoning adheres to the epistemic norms that make scientific inquiry self-correcting is poorly understood. Here, we evaluate LLM-based scientific agents across eight domains, spanning workflow execution to hypothesis-driven inquiry, through more than 25,000 agent runs and two complementary lenses: (i) a systematic performance analysis that decomposes the contributions of the base model and the agent scaffold, and (ii) a behavioral analysis of the epistemological structure of agent reasoning. We observe that the base model is the primary determinant of both performance and behavior, accounting for 41.4% of explained variance versus 1.5% for the scaffold. Across all configurations, evidence is ignored in 68% of traces, refutation-driven belief revision occurs in 26%, and convergent multi-test evidence is rare. The same reasoning pattern appears whether the agent executes a computational workflow or conducts hypothesis-driven inquiry. They persist even when agents receive near-complete successful reasoning trajectories as context, and the resulting unreliability compounds across repeated trials in epistemically demanding domains. Thus, current LLM-based agents execute scientific workflows but do not exhibit the epistemic patterns that characterize scientific reasoning. Outcome-based evaluation cannot detect these failures, and scaffold engineering alone cannot repair them. Until reasoning itself becomes a training target, the scientific knowledge produced by such agents cannot be justified by the process that generated it.

Related papers

• Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents [56.93800079330285]
  Existing multimodal benchmarks largely evaluate final-answer accuracy.<n>We introduce Sci-MMR, a benchmark for multi-step evidence-grounded scientific reasoning.<n>We find that answer accuracy consistently exceeds complete-evidence recovery rate by more than 20%.
  arXiv  Detail & Related papers  (2026-09-10T08:41:07Z)
• EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazards [73.12811119273206]
  We introduce EarthVerse, a benchmark that evaluates scientific agents through package-scoped investigations.<n>Agents inspect heterogeneous event packages, choose compatible evidence, execute transparent calculations, reconcile source differences, and preserve provenance in the final answer.<n>Across systems, the best mean answer-unit accuracy is 84.65%, while the highest Strict@95 is only 34.81%.
  arXiv  Detail & Related papers  (2026-08-24T17:29:16Z)
• Bringing analytic rigor to agentic AI for science: The Brain Researcher platform for neuroimaging data analysis [59.44063760362954]
  We present Brain Researcher, an agentic research harness operating in a neuroimaging researcher's computational environment.<n>In benchmarks, Brain Researcher increased first-choice tool-selection accuracy across seven models by 70.2 percentage points.<n>In collaborator-led and self-evolving studies, multiverse analyses exposed analytic-choice sensitivity, and scientific review classified claims as accepted, qualified, revised, blocked, rejected or deferred.
  arXiv  Detail & Related papers  (2026-08-20T11:13:27Z)
• ECLAIR: A Causally-Grounded AI Framework for Scientific Discovery in Empirical Software Engineering [51.19819833971252]
  This paper introduces ECLAIR, a causally grounded AI framework that integrates Large Language Models into every stage of the scientific process.<n>ECLAIR treats LLMs as active scientific agents operating under the principles of causal inference.
  arXiv  Detail & Related papers  (2026-08-03T14:47:03Z)
• Agentic Root Cause Analysis through Evidence-Grounded Reasoning [16.37572484400141]
  AgentRCA is a zero-shot agentic framework for evidence-grounded root cause analysis.<n>It performs inference-time reasoning by combining a data-driven digital twin with a tool-augmented large language model.<n>It achieves diagnostic performance competitive with fully supervised baselines without relying on fault-specific training.
  arXiv  Detail & Related papers  (2026-07-24T15:10:41Z)
• Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents [0.42970700836450476]
  Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery.<n>Here we propose the Hypothesis Evolution Protocol (HEP), an agent harness that provides hypothesis generation, evaluation, and evolution as explicit, auditable operations.<n>On materials-science research tasks, a HEP-equipped agent operates the hypothesis--test--evidence--belief cycle that planning-style agents lack, generalizes across research questions, and exploits the protocol more fully as the base LLM becomes more capable.
  arXiv  Detail & Related papers  (2026-07-10T08:39:30Z)
• One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution [85.65263343588026]
  We propose SAGE, a Self-correcting, Autonomous, Grounded Experimenter.<n>Its core mechanism, Multi-Hypothesis Failure Attribution (MHFA), treats recovery as a structured causal diagnosis.<n>On a 12-topic, 5-domain benchmark, SAGE increases metrics-bearing outputs from 42% to 92% over a baseline reflection.
  arXiv  Detail & Related papers  (2026-06-30T10:54:16Z)
• The Calibration Turn in AI-Assisted Research: A Conceptual and Methodological Framework for Evidence-Licensed Claims [0.9805949492148788]
  This paper develops a conceptual framework for evidence-licensed claims in AI-assisted research.<n>It represents AI-assisted research as five operators: hypothesis generation, model-mediated consequence derivation, external validation, belief update, and claim calibration.
  arXiv  Detail & Related papers  (2026-06-30T07:46:54Z)
• FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games [2.959715295394151]
  We introduce FALSIFYBENCH, an evaluation framework for hypothesis-driven reasoning inspired by the classic Wason 2-4-6 task.<n>This task captures key elements of scientific reasoning: hypothesis generation, evidence gathering, and belief revision in response to both confirming and disconfirming evidence.<n>Our evaluation of 12 LLMs across model families and scales shows that reasoning models are generally stronger scientific reasoners than instruction-tuned models, although no model comes close to optimal performance.
  arXiv  Detail & Related papers  (2026-06-03T11:33:17Z)
• Separable Pathways for Causal Reasoning: How Architectural Scaffolding Enables Hypothesis-Space Restructuring in LLM Agents [0.0]
  We extend the blicket detector paradigm from developmental science to test capacity in AI agents equipped with architectural scaffolding that targets hypothesis-space restructuring.<n>Our compositional architecture has two discrete components: context graphs, which structure exploration as typed state machines, and dynamic behaviors, which monitor for evidence that the current hypothesis space is inadequate and expand it at runtime.
  arXiv  Detail & Related papers  (2026-04-21T22:49:17Z)
• Procela: Epistemic Governance in Mechanistic Simulations Under Structural Uncertainty [0.0]
  Procela is the first framework where simulations test their own assumptions.<n>We instantiate Procela for AMR in a hospital network with three competing families.<n>Results show 20.4% error reduction and 69% cumulative regret improvement over baseline.
  arXiv  Detail & Related papers  (2026-04-01T09:22:32Z)
• Deciphering Scientific Reasoning Steps from Outcome Data for Molecule Optimization [22.850204283344524]
  We propose DESRO, a framework for deciphering scientific reasoning from outcomes.<n>By analyzing shared patterns and key differences within grouped data, a large language model (LLM) can recover the underlying logic.<n>We instantiate this framework in molecule optimization, a pivotal stage in drug discovery.
  arXiv  Detail & Related papers  (2026-03-13T06:25:27Z)
• Many AI Analysts, One Dataset: Navigating the Agentic Data Science Multiverse [22.927943525772857]
  We show that fully autonomous AI analysts built on large language models (LLMs) can reproduce a similar structured analytic diversity cheaply and at scale.<n>We show that the effects are emphsteerable: reassigning the analyst persona or LLM shifts the distribution of outcomes even after excluding methodologically deficient runs.
  arXiv  Detail & Related papers  (2026-02-21T04:10:21Z)
• Why Do AI Agents Systematically Fail at Cloud Root Cause Analysis? [1.0966260566122241]
  Failures in large-scale cloud systems incur substantial financial losses.<n>Recent efforts leverage Large Language Model (LLM) agents to automate Root Cause Analysis (RCA)<n>This paper presents a process level failure analysis of LLM-based RCA agents.
  arXiv  Detail & Related papers  (2026-02-10T16:14:05Z)
• IV Co-Scientist: Multi-Agent LLM Framework for Causal Instrumental Variable Discovery [61.15184885636171]
  In the presence of confounding between an endogenous variable and the outcome, instrumental variables (IVs) are used to isolate the causal effect of the endogenous variable.<n>We investigate whether large language models (LLMs) can aid in this task.<n>We introduce IV Co-Scientist, a multi-agent system that proposes, critiques, and refines IVs for a given treatment-outcome pair.
  arXiv  Detail & Related papers  (2026-02-08T12:28:29Z)
• FIRE-Bench: Evaluating Agents on the Rediscovery of Scientific Insights [63.32178443510396]
  We introduce FIRE-Bench (Full-cycle Insight Rediscovery Evaluation), a benchmark that evaluates agents through the rediscovery of established findings.<n>Even the strongest agents achieve limited rediscovery success (50 F1), exhibit high variance across runs, and display recurring failure modes in experimental design, execution, and evidence-based reasoning.
  arXiv  Detail & Related papers  (2026-02-02T23:21:13Z)
• Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows [203.3527268311731]
  We present an operational SGI definition grounded in the Practical Inquiry Model (PIM)<n>We operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning.<n>Our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
  arXiv  Detail & Related papers  (2025-12-18T12:44:36Z)
• Cognitive Foundations for Reasoning and Their Manifestation in LLMs [63.12951576410617]
  Large language models (LLMs) solve complex problems yet fail on simpler variants, suggesting they achieve correct outputs through mechanisms fundamentally different from human reasoning.<n>We synthesize cognitive science research into a taxonomy of 28 cognitive elements spanning reasoning invariants, meta-cognitive controls, representations for organizing reasoning & knowledge, and transformation operations.<n>We develop test-time reasoning guidance that automatically scaffold successful structures, improving performance by up to 66.7% on complex problems.
  arXiv  Detail & Related papers  (2025-11-20T18:59:00Z)
• BioDSA-1K: Benchmarking Data Science Agents for Biomedical Research [29.469867701731374]
  BioDSA-1K consists of 1,029 hypothesis-centric tasks paired with 1,177 analysis plans.<n>The benchmark enables evaluation along four axes: (1) hypothesis decision accuracy, (2) alignment between evidence and conclusion, (3) correctness of the reasoning process, and (4) executability of the AI-generated analysis code.
  arXiv  Detail & Related papers  (2025-05-22T01:02:21Z)
• Context-Aware Reasoning On Parametric Knowledge for Inferring Causal Variables [49.31233968546582]
  We introduce a novel benchmark where the objective is to complete a partial causal graph.<n>We show the strong ability of LLMs to hypothesize the backdoor variables between a cause and its effect.<n>Unlike simple memorization of fixed associations, our task requires the LLM to reason according to the context of the entire graph.
  arXiv  Detail & Related papers  (2024-09-04T10:37:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.