Finetuning with Scientific Data Increases Hallucinations: A Multi-domain Factuality Evaluation of LLMs

• URL: http://arxiv.org/abs/2606.21359v1
• Date: Fri, 19 Jun 2026 12:03:02 GMT
• Title: Finetuning with Scientific Data Increases Hallucinations: A Multi-domain Factuality Evaluation of LLMs
• Abstract summary: Large language models (LLMs) are increasingly used to communicate and explain scientific concepts.<n>We evaluate 18 LLMs by comparing each scientifically fine-tuned model against its general-purpose base.
• Score: 8.700507644617964
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses significant risks in this high stakes use-case. Prior hallucination evaluation work remains largely restricted to the biomedical domain, treats hallucination as a binary task, and has not examined the growing family of scientifically fine-tuned LLMs. We address these gaps with SciFactCheck, a benchmark of 2,500 prompts across five scientific domains, paired with a modular evaluation framework targeting three factuality hallucination types: unverifiability, overclaim, and attribution. Using a controlled minimal-pairing design, we evaluate 18 LLMs by comparing each scientifically fine-tuned model against its general-purpose base. Our results indicate that 1. Scientifically fine-tuned models exhibit degraded factual reliability across all hallucination types and scientific domains, and 2. Fine-tuned models are internally less confident yet linguistically more assertive. A human pilot study further reveals that current fact-checking tools show only modest agreement with expert judgments on scientific content, and that defining scientifically check-worthy claims remains contested even among human annotators. Our findings fundamentally challenge current methods of domain-specific fine-tuning for factuality and call for developing improved verification infrastructure for scientific content.

Related papers

• Can LLMs Discover Scientific Laws in Real and Parallel Worlds? [58.76147897928744]
  We introduce SCILAWS-BENCH, a benchmark for scientific law discovery built from published research and real scientific data.<n>It comprises 118 problems drawn from 381 scientific papers, covering 291 candidate laws and roughly 8M real data points across six scientific disciplines.<n>We find that predictive fit can diverge from scientific validity, memorization shapes whether models reproduce or move beyond published formulas, and our best-of-N study reveals a selection bottleneck.
  arXiv  Detail & Related papers  (2026-09-01T17:17:59Z)
• Science Edge Evaluation: SEE the Missing Step Toward Real Scientific Discovery [41.32793603340132]
  Large language models (LLMs) are increasingly involved in scientific discovery.<n>It remains unclear whether they can support complex real laboratory science.<n> Evaluation of 19 large language models (MLLMs) shows that even the best-performing model reaches only 48.7% accuracy.
  arXiv  Detail & Related papers  (2026-08-07T08:06:31Z)
• KnowHal: A Knowledge-Driven Benchmark for Comprehensive Multimodal Hallucination Evaluation [9.739392718456843]
  Knowledge-related failures are often investigated separately, lacking a unified evaluation framework across different hallucination dimensions.<n>We propose KnowHal, a benchmark that explicitly incorporates knowledge hallucination into multimodal hallucination evaluation spanning four dimensions.<n>KnowHal constructs paired positive and negative questions over shared images and entities, enabling controlled comparisons among perceptual errors, knowledge-related errors, and false-premise acceptance.
  arXiv  Detail & Related papers  (2026-08-04T15:03:29Z)
• MeasHalu: Mitigation of Scientific Measurement Hallucinations for Large Language Models with Enhanced Reasoning [16.93649712632818]
  MeasHalu is a framework for mitigating scientific measurement hallucinations.<n>MeasHalu substantially reduces hallucination rates and improves overall accuracy on the MeasEval benchmark.
  arXiv  Detail & Related papers  (2026-04-18T09:26:52Z)
• M2-Verify: A Large-Scale Multidomain Benchmark for Checking Multimodal Claim Consistency [17.049806985851994]
  M2-Verify is a large-scale multimodal dataset for checking scientific claim consistency.<n>It provides over 469K instances across 16 domains, rigorously validated through expert audits.
  arXiv  Detail & Related papers  (2026-04-01T18:18:10Z)
• Knowing When Not to Answer: Abstention-Aware Scientific Reasoning [2.680633756465714]
  In scientific settings, unsupported or uncertain conclusions can be more harmful than abstaining.<n>We study this problem through an abstention-aware verification framework.<n>We evaluate this framework across two scientific benchmarks: SciFact and PubMedQA.
  arXiv  Detail & Related papers  (2026-02-15T15:29:43Z)
• Evaluating Large Language Models in Scientific Discovery [91.732562776782]
  Large language models (LLMs) are increasingly applied to scientific research, yet prevailing science benchmarks probe decontextualized knowledge.<n>We introduce a scenario-grounded benchmark that evaluates LLMs across biology, chemistry, materials, and physics.<n>The framework assesses models at two levels: (i) question-level accuracy on scenario-tied items and (ii) project-level performance.
  arXiv  Detail & Related papers  (2025-12-17T16:20:03Z)
• General-purpose AI models can generate actionable knowledge on agroecological crop protection [0.0]
  We verify scientific knowledge on agroecological crop protection generated by web-grounded or non-grounded large language models.<n>DeepSeek consistently screened a 4.8-49.7-fold larger literature corpus and reported 1.6-2.4-fold more biological control agents or management solutions than ChatGPT.
  arXiv  Detail & Related papers  (2025-12-12T11:17:13Z)
• A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers [251.23085679210206]
  Scientific Large Language Models (Sci-LLMs) are transforming how knowledge is represented, integrated, and applied in scientific research.<n>This survey reframes the development of Sci-LLMs as a co-evolution between models and their underlying data substrate.<n>We formulate a unified taxonomy of scientific data and a hierarchical model of scientific knowledge.
  arXiv  Detail & Related papers  (2025-08-28T18:30:52Z)
• MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models [82.30696225661615]
  We introduce MedHallu, the first benchmark specifically designed for medical hallucination detection.<n>We show that state-of-the-art LLMs, including GPT-4o, Llama-3.1, and medically fine-tuned UltraMedical, struggle with this binary hallucination detection task.<n>Using bidirectional entailment clustering, we show that harder-to-detect hallucinations are semantically closer to ground truth.
  arXiv  Detail & Related papers  (2025-02-20T06:33:23Z)
• A Debate-Driven Experiment on LLM Hallucinations and Accuracy [7.821303946741665]
  This study investigates the phenomenon of hallucination in large language models (LLMs) Multiple instances of GPT-4o-Mini models engage in a debate-like interaction prompted with questions from the TruthfulQA dataset. One model is deliberately instructed to generate plausible but false answers while the other models are asked to respond truthfully.
  arXiv  Detail & Related papers  (2024-10-25T11:41:27Z)
• LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery [141.39722070734737]
  We propose to enhance the knowledge-driven, abstract reasoning abilities of Large Language Models with the computational strength of simulations. We introduce Scientific Generative Agent (SGA), a bilevel optimization framework. We conduct experiments to demonstrate our framework's efficacy in law discovery and molecular design.
  arXiv  Detail & Related papers  (2024-05-16T03:04:10Z)
• Language Models Hallucinate, but May Excel at Fact Verification [89.0833981569957]
  Large language models (LLMs) frequently "hallucinate," resulting in non-factual outputs. Even GPT-3.5 produces factual outputs less than 25% of the time. This underscores the importance of fact verifiers in order to measure and incentivize progress.
  arXiv  Detail & Related papers  (2023-10-23T04:39:01Z)
• Large Language Models for Automated Open-domain Scientific Hypotheses Discovery [50.40483334131271]
  This work proposes the first dataset for social science academic hypotheses discovery. Unlike previous settings, the new dataset requires (1) using open-domain data (raw web corpus) as observations; and (2) proposing hypotheses even new to humanity. A multi- module framework is developed for the task, including three different feedback mechanisms to boost performance.
  arXiv  Detail & Related papers  (2023-09-06T05:19:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.