Sparks of Science: Hypothesis Generation Using Structured Paper Data

• URL: http://arxiv.org/abs/2504.12976v1
• Date: Thu, 17 Apr 2025 14:29:18 GMT
• Title: Sparks of Science: Hypothesis Generation Using Structured Paper Data
• Authors: Charles O'Neill, Tirthankar Ghosal, Roberta Răileanu, Mike Walmsley, Thang Bui, Kevin Schawinski, Ioana Ciucă,
• Abstract summary: We introduce HypoGen, the first dataset of approximately 5500 structured problem-hypothesis pairs extracted from top-tier computer science conferences.<n>We demonstrate that framing hypothesis generation as conditional language modelling, with the model fine-tuned on Bit-Flip-Spark and the Chain-of-Reasoning.<n>We show that by fine-tuning on our HypoGen dataset we improve the novelty, feasibility, and overall quality of the generated hypotheses.
• Score: 1.250723303641055
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generating novel and creative scientific hypotheses is a cornerstone in achieving Artificial General Intelligence. Large language and reasoning models have the potential to aid in the systematic creation, selection, and validation of scientifically informed hypotheses. However, current foundation models often struggle to produce scientific ideas that are both novel and feasible. One reason is the lack of a dedicated dataset that frames Scientific Hypothesis Generation (SHG) as a Natural Language Generation (NLG) task. In this paper, we introduce HypoGen, the first dataset of approximately 5500 structured problem-hypothesis pairs extracted from top-tier computer science conferences structured with a Bit-Flip-Spark schema, where the Bit is the conventional assumption, the Spark is the key insight or conceptual leap, and the Flip is the resulting counterproposal. HypoGen uniquely integrates an explicit Chain-of-Reasoning component that reflects the intellectual process from Bit to Flip. We demonstrate that framing hypothesis generation as conditional language modelling, with the model fine-tuned on Bit-Flip-Spark and the Chain-of-Reasoning (and where, at inference, we only provide the Bit), leads to improvements in the overall quality of the hypotheses. Our evaluation employs automated metrics and LLM judge rankings for overall quality assessment. We show that by fine-tuning on our HypoGen dataset we improve the novelty, feasibility, and overall quality of the generated hypotheses. The HypoGen dataset is publicly available at huggingface.co/datasets/UniverseTBD/hypogen-dr1.

Related papers

• HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generation [24.656083479331645]
  We introduce HypoBench, a novel benchmark designed to evaluate hypothesis generation methods across multiple aspects.<n>We evaluate four state-of-the-art LLMs combined with six existing hypothesis-generation methods.<n>Results indicate that there is still significant room for improvement, as current hypothesis generation methods do not fully uncover all relevant or meaningful patterns.
  arXiv  Detail & Related papers  (2025-04-15T18:00:00Z)
• Hypothesizing Missing Causal Variables with LLMs [55.28678224020973]
  We formulate a novel task where the input is a partial causal graph with missing variables, and the output is a hypothesis about the missing variables to complete the partial graph. We show the strong ability of LLMs to hypothesize the mediation variables between a cause and its effect. We also observe surprising results where some of the open-source models outperform the closed GPT-4 model.
  arXiv  Detail & Related papers  (2024-09-04T10:37:44Z)
• Graph Stochastic Neural Process for Inductive Few-shot Knowledge Graph Completion [63.68647582680998]
  We focus on a task called inductive few-shot knowledge graph completion (I-FKGC) Inspired by the idea of inductive reasoning, we cast I-FKGC as an inductive reasoning problem. We present a neural process-based hypothesis extractor that models the joint distribution of hypothesis, from which we can sample a hypothesis for predictions. In the second module, based on the hypothesis, we propose a graph attention-based predictor to test if the triple in the query set aligns with the extracted hypothesis.
  arXiv  Detail & Related papers  (2024-08-03T13:37:40Z)
• SPIN: SE(3)-Invariant Physics Informed Network for Binding Affinity Prediction [3.406882192023597]
  Accurate prediction of protein-ligand binding affinity is crucial for drug development. Traditional methods often fail to accurately model the complex's spatial information. We propose SPIN, a model that incorporates various inductive biases applicable to this task.
  arXiv  Detail & Related papers  (2024-07-10T08:40:07Z)
• Hypothesis Generation with Large Language Models [28.73562677221476]
  We focus on hypothesis generation based on data (i.e., labeled examples)<n>Inspired by multi-armed bandits, we design a reward function to inform the exploitation-exploration tradeoff in the update process.<n>Our algorithm is able to generate hypotheses that enable much better predictive performance than few-shot prompting in classification tasks.
  arXiv  Detail & Related papers  (2024-04-05T18:00:07Z)
• Large Language Models for Automated Open-domain Scientific Hypotheses Discovery [50.40483334131271]
  This work proposes the first dataset for social science academic hypotheses discovery. Unlike previous settings, the new dataset requires (1) using open-domain data (raw web corpus) as observations; and (2) proposing hypotheses even new to humanity. A multi- module framework is developed for the task, including three different feedback mechanisms to boost performance.
  arXiv  Detail & Related papers  (2023-09-06T05:19:41Z)
• On the Joint Interaction of Models, Data, and Features [82.60073661644435]
  We introduce a new tool, the interaction tensor, for empirically analyzing the interaction between data and model through features. Based on these observations, we propose a conceptual framework for feature learning. Under this framework, the expected accuracy for a single hypothesis and agreement for a pair of hypotheses can both be derived in closed-form.
  arXiv  Detail & Related papers  (2023-06-07T21:35:26Z)
• SciMON: Scientific Inspiration Machines Optimized for Novelty [68.46036589035539]
  We explore and enhance the ability of neural language models to generate novel scientific directions grounded in literature. We take a dramatic departure with a novel setting in which models use as input background contexts. We present SciMON, a modeling framework that uses retrieval of "inspirations" from past scientific papers.
  arXiv  Detail & Related papers  (2023-05-23T17:12:08Z)
• Neural Causal Models for Counterfactual Identification and Estimation [62.30444687707919]
  We study the evaluation of counterfactual statements through neural models. First, we show that neural causal models (NCMs) are expressive enough. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions.
  arXiv  Detail & Related papers  (2022-09-30T18:29:09Z)
• Learning to Reason for Text Generation from Scientific Tables [100.61286775597947]
  We introduce SciGen, a new challenge dataset for the task of reasoning-aware data-to-text generation. Describing scientific tables goes beyond the surface realization of the table content and requires reasoning over table values. We study the effectiveness of state-of-the-art data-to-text generation models on SciGen and evaluate the results using common metrics as well as human evaluation.
  arXiv  Detail & Related papers  (2021-04-16T18:01:36Z)
• Exploring Lexical Irregularities in Hypothesis-Only Models of Natural Language Inference [5.283529004179579]
  Natural Language Inference (NLI) or Recognizing Textual Entailment (RTE) is the task of predicting the entailment relation between a pair of sentences. Models that understand entailment should encode both, the premise and the hypothesis. Experiments by Poliak et al. revealed a strong preference of these models towards patterns observed only in the hypothesis.
  arXiv  Detail & Related papers  (2021-01-19T01:08:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.