Hypothesis generation and updating in large language models

• URL: http://arxiv.org/abs/2605.05851v1
• Date: Thu, 07 May 2026 08:24:54 GMT
• Title: Hypothesis generation and updating in large language models
• Abstract summary: Large language models (LLMs) increasingly help people solve problems, but how they perform inference remains unclear.<n>We study this question in the number game, a controlled setting in which a learner infers the hypothesis supported by a few positive integers.<n>We measure the posterior over hypotheses using three complementary probes: posterior prediction, hypothesis evaluation, and hypothesis generation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models (LLMs) increasingly help people solve problems, from debugging code to repairing machinery. This process requires generating plausible hypotheses from partial descriptions, then updating them as more information arrives. Yet how LLMs perform this form of inference, and how close it is to optimal, remains unclear. We study this question in the number game, a controlled setting in which a learner infers the hypothesis supported by a few positive integers, such as $\{16, 8, 2, 64\}$: a rule like powers of 2 or an interval like numbers near 20. We measure the posterior over hypotheses using three complementary probes: posterior prediction, hypothesis evaluation, and hypothesis generation. We then compare LLM behavior with an optimal Bayesian model and human behavior, and test whether the same posterior is expressed across probes. LLMs are often well described by a two-parameter Bayesian fit, but with systematic offsets: by default they show a strong-sampling assumption that creates an implicit Occam's razor, favoring narrower hypotheses, while thinking mode shifts them toward greater prior reliance. We also find a robust evaluation--generation gap: LLMs select more correct hypotheses during hypothesis evaluation but generate simpler, more rule-like hypotheses. Finally, this Bayesian-with-bias pattern does not extrapolate. Models can behave as if they hold rule-like hypotheses over observed examples, yet generalize poorly to parts of the hypothesis domain not covered by those examples. Our results highlight a limitation of LLMs as general problem solvers, especially for scientific inference, where hypotheses must go beyond the data.

Related papers

• NxN E-valuation: Hypothesis Certification via a Conformal CRT Null [7.215201796507224]
  We propose NxN E-valuation, a handy, e-value-based hypothesis-certification algorithm.<n>It lets a hypothesis be verified without building any case-specific certification procedure.<n>NxN E-valuation is especially suited to LLM-based exploration systems.
  arXiv  Detail & Related papers  (2026-08-06T22:14:53Z)
• Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypotheses [8.864909622103388]
  Verbalized Machine Learning (VML) parameterizes a model as a natural-language prompt that an LLM evaluates as f(x; theta)<n>We propose the Verbalized Particle Posterior (VPP), which treats verbalized learning as a Bayesian inference problem.
  arXiv  Detail & Related papers  (2026-07-25T00:04:56Z)
• Before the Action: Benchmarking LLMs on Prospective Hypothesis Discovery [95.44432473367836]
  Large language models (LLMs) excel at answering pre-specified questions, yet their ability to navigate the open-ended, pre-conclusion stage of discovery remains largely unmeasured.<n>We introduce Prospective Hypothesis Discovery (PHD), which asks models to autonomously construct grounded, discnative, and testable hypothesis spaces from inconclusive evidence.<n>We introduce HypoArena, comprising HypoData, a benchmark of 988 cases across six scientific and analytical domains, and HypoEval, an evaluation framework for open-ended hypothesis sets.
  arXiv  Detail & Related papers  (2026-07-17T08:56:43Z)
• HypoSpace: Evaluating LLM Creativity as Set-Valued Hypothesis Generators under Underdetermination [46.896452542901805]
  We introduce HypoSpace, a diagnostic suite that treats LLMs as samplers of finite hypothesis sets.<n>We instantiate HypoSpace in three structured domains with deterministic validators and exactly enumerated hypothesis spaces.<n>Across instruction-tuned and reasoning-focused models, Validity often remains high while Uniqueness and Recovery degrade as the admissible space grows.
  arXiv  Detail & Related papers  (2025-10-17T13:00:32Z)
• MOOSE-Chem2: Exploring LLM Limits in Fine-Grained Scientific Hypothesis Discovery via Hierarchical Search [102.11776494401705]
  Large language models (LLMs) have shown promise in automating scientific hypothesis generation.<n>Existing approaches primarily yield coarse-grained hypotheses lacking critical methodological and experimental details.<n>We introduce and formally define the new task of fine-grained scientific hypothesis discovery.
  arXiv  Detail & Related papers  (2025-05-25T16:13:46Z)
• Resolving Multiple-Dynamic Model Uncertainty in Hypothesis-Driven Belief-MDPs [4.956709222278243]
  We present a hypothesis-driven belief MDP that enables reasoning over multiple hypotheses. We also present a new belief MDP that balances the goals of determining the (most likely) correct hypothesis and performing well in the underlying POMDP.
  arXiv  Detail & Related papers  (2024-11-21T18:36:19Z)
• A Hypothesis-Driven Framework for the Analysis of Self-Rationalising Models [0.8702432681310401]
  We use a Bayesian network to implement a hypothesis about how a task is solved. The resulting models do not exhibit a strong similarity to GPT-3.5. We discuss the implications of this as well as the framework's potential to approximate LLM decisions better in future work.
  arXiv  Detail & Related papers  (2024-02-07T12:26:12Z)
• Source-Free Unsupervised Domain Adaptation with Hypothesis Consolidation of Prediction Rationale [53.152460508207184]
  Source-Free Unsupervised Domain Adaptation (SFUDA) is a challenging task where a model needs to be adapted to a new domain without access to target domain labels or source domain data. This paper proposes a novel approach that considers multiple prediction hypotheses for each sample and investigates the rationale behind each hypothesis. To achieve the optimal performance, we propose a three-step adaptation process: model pre-adaptation, hypothesis consolidation, and semi-supervised learning.
  arXiv  Detail & Related papers  (2024-02-02T05:53:22Z)
• Optimal Multi-Distribution Learning [88.3008613028333]
  Multi-distribution learning seeks to learn a shared model that minimizes the worst-case risk across $k$ distinct data distributions.<n>We propose a novel algorithm that yields an varepsilon-optimal randomized hypothesis with a sample complexity on the order of (d+k)/varepsilon2.
  arXiv  Detail & Related papers  (2023-12-08T16:06:29Z)
• On Localized Discrepancy for Domain Adaptation [146.4580736832752]
  This paper studies the localized discrepancies defined on the hypothesis space after localization. Their values will be different if we exchange the two domains, thus can reveal asymmetric transfer difficulties.
  arXiv  Detail & Related papers  (2020-08-14T08:30:02Z)
• Learning programs by learning from failures [26.955785230358963]
  We describe an inductive logic programming (ILP) approach called learning from failures. In this approach, an ILP system decomposes the learning problem into three separate stages: generate, test, and constrain. We introduce Popper, an ILP system that implements this approach by combining answer set programming and Prolog.
  arXiv  Detail & Related papers  (2020-05-05T14:55:07Z)
• Explaining Question Answering Models through Text Generation [42.36596190720944]
  Large pre-trained language models (LMs) have been shown to perform surprisingly well when fine-tuned on tasks that require commonsense and world knowledge. It is difficult to explain what is the knowledge in the LM that allows it to make a correct prediction in end-to-end architectures. We show on several tasks that our model reaches performance that is comparable to end-to-end architectures.
  arXiv  Detail & Related papers  (2020-04-12T09:06:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.