Can LLMs extract scientific consensus? A case study in high-temperature superconductivity

• URL: http://arxiv.org/abs/2606.07570v1
• Date: Tue, 26 May 2026 03:52:37 GMT
• Title: Can LLMs extract scientific consensus? A case study in high-temperature superconductivity
• Abstract summary: Large language models (LLMs) have shown impressive performance in information extraction and summarization.<n>We investigate this problem in the context of high-temperature superconductivity (HTS), a long-standing and highly debated topic in condensed matter physics.
• Score: 8.052791451171487
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific knowledge is increasingly dispersed across vast and heterogeneous scientific literature, where important claims are often implicit, evolving, and internally debated. While large language models (LLMs) have shown impressive performance in information extraction and summarization, their ability to recover latent scientific consensus remains unclear. Here, we investigate this problem in the context of high-temperature superconductivity (HTS), a long-standing and highly debated topic in condensed matter physics, as a challenging testbed. Using near 18,000 highly-cited publications over the past seven decades, we construct a structured knowledge graph linking competing superconducting mechanisms, material families, evidential modalities, and citation relations. We find that LLM-extracted representations recover coherent and physically interpretable structures, including family-dependent mechanism profiles, evidence-specific correlations, and citation-mediated temporal evolution of scientific beliefs. Ablation studies on LLM further show that the global structure remains robust across prompting, decoding, and model variations. Our results suggest that LLMs can indeed serve as scalable tools for deciphering scientific knowledge in domains characterized by competing interpretations and evolving knowledge.

Related papers

• Can LLMs Discover Scientific Laws in Real and Parallel Worlds? [58.76147897928744]
  We introduce SCILAWS-BENCH, a benchmark for scientific law discovery built from published research and real scientific data.<n>It comprises 118 problems drawn from 381 scientific papers, covering 291 candidate laws and roughly 8M real data points across six scientific disciplines.<n>We find that predictive fit can diverge from scientific validity, memorization shapes whether models reproduce or move beyond published formulas, and our best-of-N study reveals a selection bottleneck.
  arXiv  Detail & Related papers  (2026-09-01T17:17:59Z)
• SciMIF: Understanding Multimodal Instruction Following in Scientific Domains [50.54744513908656]
  We introduce SciMIF, a novel benchmark designed to evaluate the capability of MLLMs in following complex scientific instructions.<n>Based on an analysis of 22 distinct tasks across 5 representative scientific disciplines, we propose a comprehensive taxonomy.<n>We develop a high-fidelity instruction injection pipeline to systematically augment existing scientific datasets.
  arXiv  Detail & Related papers  (2026-08-26T16:30:20Z)
• Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discovery [41.32793603340132]
  Large language models (LLMs) are increasingly involved in scientific discovery.<n>It remains unclear whether they can support complex real laboratory science.<n> Evaluation of 19 large language models (MLLMs) shows that even the best-performing model reaches only 48.7% accuracy.
  arXiv  Detail & Related papers  (2026-08-07T08:06:31Z)
• A Multi-Agent Human-LLM Collaborative Framework for Closed-Loop Scientific Literature Summarization [14.654160505140426]
  We introduce Elhuyar, a multi-agent, human-in-the-loop system that integrates AI, large language models, and human scientists.<n>The framework distributes tasks among specialized agents for filtering papers, extracting data, fitting models, and summarizing findings, with human oversight ensuring reliability.<n>It generates structured reports with extracted data, visualizations, model equations, and text summaries, enabling deeper inquiry through iterative refinement.
  arXiv  Detail & Related papers  (2026-04-01T22:58:42Z)
• How well can off-the-shelf LLMs elucidate molecular structures from mass spectra using chain-of-thought reasoning? [51.286853421822705]
  Large language models (LLMs) have shown promise for reasoning-intensive scientific tasks, but their capability for chemical interpretation is still unclear.<n>We introduce a Chain-of-Thought (CoT) prompting framework and benchmark that evaluate how LLMs reason about mass spectral data to predict molecular structures.<n>Our evaluation across metrics of SMILES validity, formula consistency, and structural similarity reveals that while LLMs can produce syntactically valid and partially plausible structures, they fail to achieve chemical accuracy or link reasoning to correct molecular predictions.
  arXiv  Detail & Related papers  (2026-01-09T20:08:42Z)
• What's In Your Field? Mapping Scientific Research with Knowledge Graphs and Large Language Models [4.8261605642238745]
  Large language models (LLMs) fail to capture detailed relationships across large bodies of work.<n>Structured representations offer a natural complement -- enabling systematic analysis across the whole corpus.<n>We prototype a system that answers precise questions about the literature as a whole.
  arXiv  Detail & Related papers  (2025-03-12T23:24:40Z)
• Improving Scientific Hypothesis Generation with Knowledge Grounded Large Language Models [20.648157071328807]
  Large language models (LLMs) can identify novel research directions by analyzing existing knowledge. LLMs are prone to generating hallucinations'', outputs that are plausible-sounding but factually incorrect. We propose KG-CoI, a system that enhances LLM hypothesis generation by integrating external, structured knowledge from knowledge graphs.
  arXiv  Detail & Related papers  (2024-11-04T18:50:00Z)
• A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery [68.48094108571432]
  Large language models (LLMs) have revolutionized the way text and other modalities of data are handled. We aim to provide a more holistic view of the research landscape by unveiling cross-field and cross-modal connections between scientific LLMs.
  arXiv  Detail & Related papers  (2024-06-16T08:03:24Z)
• LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery [141.39722070734737]
  We propose to enhance the knowledge-driven, abstract reasoning abilities of Large Language Models with the computational strength of simulations. We introduce Scientific Generative Agent (SGA), a bilevel optimization framework. We conduct experiments to demonstrate our framework's efficacy in law discovery and molecular design.
  arXiv  Detail & Related papers  (2024-05-16T03:04:10Z)
• Scientific Large Language Models: A Survey on Biological & Chemical Domains [47.97810890521825]
  Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension. The application of LLMs extends beyond conventional linguistic boundaries, encompassing specialized linguistic systems developed within various scientific disciplines. As a burgeoning area in the community of AI for Science, scientific LLMs warrant comprehensive exploration.
  arXiv  Detail & Related papers  (2024-01-26T05:33:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.