Open-ended Scientific Discovery via Bayesian Surprise

• URL: http://arxiv.org/abs/2507.00310v1
• Date: Mon, 30 Jun 2025 22:53:59 GMT
• Title: Open-ended Scientific Discovery via Bayesian Surprise
• Abstract summary: AutoDS is a method for open-ended scientific discovery that instead drives scientific exploration using Bayesian surprise.<n>We evaluate AutoDS in the setting of data-driven discovery across 21 real-world datasets spanning domains such as biology, economics, finance, and behavioral science.
• Score: 63.26412847240136
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The promise of autonomous scientific discovery (ASD) hinges not only on answering questions, but also on knowing which questions to ask. Most recent works in ASD explore the use of large language models (LLMs) in goal-driven settings, relying on human-specified research questions to guide hypothesis generation. However, scientific discovery may be accelerated further by allowing the AI system to drive exploration by its own criteria. The few existing approaches in open-ended ASD select hypotheses based on diversity heuristics or subjective proxies for human interestingness, but the former struggles to meaningfully navigate the typically vast hypothesis space, and the latter suffers from imprecise definitions. This paper presents AutoDS -- a method for open-ended ASD that instead drives scientific exploration using Bayesian surprise. Here, we quantify the epistemic shift from the LLM's prior beliefs about a hypothesis to its posterior beliefs after gathering experimental results. To efficiently explore the space of nested hypotheses, our method employs a Monte Carlo tree search (MCTS) strategy with progressive widening using surprisal as the reward function. We evaluate AutoDS in the setting of data-driven discovery across 21 real-world datasets spanning domains such as biology, economics, finance, and behavioral science. Our results demonstrate that under a fixed budget, AutoDS substantially outperforms competitors by producing 5--29\% more discoveries deemed surprising by the LLM. Our human evaluation further finds that two-thirds of AutoDS discoveries are surprising to the domain experts, suggesting this is an important step forward towards building open-ended ASD systems.

Related papers

• Before the Action: Benchmarking LLMs on Prospective Hypothesis Discovery [95.44432473367836]
  Large language models (LLMs) excel at answering pre-specified questions, yet their ability to navigate the open-ended, pre-conclusion stage of discovery remains largely unmeasured.<n>We introduce Prospective Hypothesis Discovery (PHD), which asks models to autonomously construct grounded, discnative, and testable hypothesis spaces from inconclusive evidence.<n>We introduce HypoArena, comprising HypoData, a benchmark of 988 cases across six scientific and analytical domains, and HypoEval, an evaluation framework for open-ended hypothesis sets.
  arXiv  Detail & Related papers  (2026-07-17T08:56:43Z)
• Autonomous Scientific Discovery via Iterative Meta-Reflection [44.46230654454998]
  We introduce DiscoPER, a framework that conducts open-ended research by dynamically generating and executing code to explore datasets without pre-specified research objectives.<n>By treating prior discoveries as empirical data, DiscoPER actively redirecting hypothesis exploration toward uncharted regions of the search space.<n> evaluated on iNatDisco, a new multimodal ecological knowledge benchmark with pattern-level ground truth obtained from peer-reviewed literature.
  arXiv  Detail & Related papers  (2026-07-01T16:16:07Z)
• Evidence-Informed LLM Beliefs for Continual Scientific Discovery [66.36520571709482]
  We introduce two complementary changes to the original search procedure: belief-update filtering and diversity.<n>We find that embedding-based retrieval-augmented generation over prior discoveries best anticipates eventual posteriors, identifying 37.5% of static surprisals as spurious.<n>We then modify search to avoid these spurious rewards and prioritize hypotheses that remain surprising under non-stationary beliefs.
  arXiv  Detail & Related papers  (2026-06-28T04:06:12Z)
• FIRE-Bench: Evaluating Agents on the Rediscovery of Scientific Insights [63.32178443510396]
  We introduce FIRE-Bench (Full-cycle Insight Rediscovery Evaluation), a benchmark that evaluates agents through the rediscovery of established findings.<n>Even the strongest agents achieve limited rediscovery success (50 F1), exhibit high variance across runs, and display recurring failure modes in experimental design, execution, and evidence-based reasoning.
  arXiv  Detail & Related papers  (2026-02-02T23:21:13Z)
• Hypothesis Hunting with Evolving Networks of Autonomous Scientific Agents [52.50038914857797]
  We term this process hypothesis hunting: the cumulative search for insight through sustained exploration across vast and complex hypothesis spaces.<n>We introduce AScience, a framework modeling discovery as the interaction of agents, networks, and evaluation norms, and implement it as ASCollab.<n> Experiments show that such social dynamics enable the accumulation of expert-rated results along the diversity-quality-novelty frontier.
  arXiv  Detail & Related papers  (2025-10-08T08:47:07Z)
• HARPA: A Testability-Driven, Literature-Grounded Framework for Research Ideation [29.9491787481972]
  HARPA is a tool to generate hypotheses that are both testable and grounded in the scientific literature.<n>Our evaluations show that HARPA-generated hypothesis-driven research proposals perform comparably to a strong baseline AI-researcher.<n>When tested with the ASD agent (CodeScientist), HARPA produced more successful executions (20 vs. 11 out of 40) and fewer failures (16 vs. 21 out of 40)
  arXiv  Detail & Related papers  (2025-10-01T07:52:19Z)
• Beyond Optimization: Exploring Novelty Discovery in Autonomous Experiments [0.8086551202409836]
  We introduce a novel framework, INS2ANE, to enhance the discovery of novel phenomena in autonomous experimentation.<n>Our method integrates two key components: (1) a novelty scoring system that evaluates the uniqueness of experimental results, and (2) a strategic sampling mechanism that promotes exploration of under-sampled regions.
  arXiv  Detail & Related papers  (2025-08-27T20:19:04Z)
• HypoChainer: A Collaborative System Combining LLMs and Knowledge Graphs for Hypothesis-Driven Scientific Discovery [4.020865072189471]
  We propose HypoChainer, a visualization framework that integrates human expertise, knowledge graphs, and reasoning.<n> HypoChainer operates in three stages: First, exploration and contextualization -- experts use retrieval-augmented LLMs (RAGs) and dimensionality reduction.<n>Second, hypothesis chain formation -- experts iteratively examine KG relationships around predictions and semantically linked entities.<n>Third, validation prioritization -- refined hypotheses are filtered based on KG-supported evidence to identify high-priority candidates for experimentation.
  arXiv  Detail & Related papers  (2025-07-23T05:02:54Z)
• MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search [93.64235254640967]
  Large language models (LLMs) have shown promise in automating scientific hypothesis generation.<n>We define the novel task of fine-grained scientific hypothesis discovery.<n>We propose a hierarchical search method that incrementally proposes and integrates details into the hypothesis.
  arXiv  Detail & Related papers  (2025-05-25T16:13:46Z)
• IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery [27.218896203253987]
  IRIS is an open-source platform designed for researchers to leverage large language models (LLMs)-assisted scientific ideation.<n>IRIS incorporates innovative features to enhance ideation, including adaptive test-time compute expansion via Monte Carlo Tree Search (MCTS), fine-grained feedback mechanism, and query-based literature synthesis.<n>We conduct a user study with researchers across diverse disciplines, validating the effectiveness of our system in enhancing ideation.
  arXiv  Detail & Related papers  (2025-04-23T14:01:36Z)
• ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decomposition [67.26124739345332]
  Large language models (LLMs) have demonstrated potential in assisting scientific research, yet their ability to discover high-quality research hypotheses remains unexamined.<n>We introduce the first large-scale benchmark for evaluating LLMs with a near-sufficient set of sub-tasks of scientific discovery.<n>We develop an automated framework that extracts critical components - research questions, background surveys, inspirations, and hypotheses - from scientific papers.
  arXiv  Detail & Related papers  (2025-03-27T08:09:15Z)
• CodeScientist: End-to-End Semi-Automated Scientific Discovery with Code-based Experimentation [48.12054700748627]
  We introduce CodeScientist, a novel ASD system that frames ideation and experiment construction as a form of genetic search jointly.<n>We use this paradigm to conduct hundreds of automated experiments on machine-generated ideas broadly in the domain of agents and virtual environments.
  arXiv  Detail & Related papers  (2025-03-20T22:37:17Z)
• Can Large Language Models Help Experimental Design for Causal Discovery? [94.66802142727883]
  Large Language Model Guided Intervention Targeting (LeGIT) is a robust framework that effectively incorporates LLMs to augment existing numerical approaches for the intervention targeting in causal discovery.<n>LeGIT demonstrates significant improvements and robustness over existing methods and even surpasses humans.
  arXiv  Detail & Related papers  (2025-03-03T03:43:05Z)
• Large Language Models are Zero Shot Hypothesis Proposers [17.612235393984744]
  Large Language Models (LLMs) hold a wealth of global and interdisciplinary knowledge that promises to break down information barriers. We construct a dataset consist of background knowledge and hypothesis pairs from biomedical literature. We evaluate the hypothesis generation capabilities of various top-tier instructed models in zero-shot, few-shot, and fine-tuning settings.
  arXiv  Detail & Related papers  (2023-11-10T10:03:49Z)
• Large Language Models for Automated Open-domain Scientific Hypotheses Discovery [50.40483334131271]
  This work proposes the first dataset for social science academic hypotheses discovery. Unlike previous settings, the new dataset requires (1) using open-domain data (raw web corpus) as observations; and (2) proposing hypotheses even new to humanity. A multi- module framework is developed for the task, including three different feedback mechanisms to boost performance.
  arXiv  Detail & Related papers  (2023-09-06T05:19:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.