Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning

• URL: http://arxiv.org/abs/2608.20960v1
• Date: Fri, 21 Aug 2026 10:27:49 GMT
• Title: Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning
• Abstract summary: Language models (LMs) are trained on static scientific corpora, whereas scientific knowledge continuously evolves through correction and revision.<n>This creates a need for LMs to forget obsolete scientific claims.<n>We show that current unlearning approaches are unable to effectively eliminate claim-level knowledge and often achieve only superficial suppression.
• Score: 47.831933174173464
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language models (LMs) are trained on static scientific corpora, whereas scientific knowledge continuously evolves through correction and revision. Scientific claims encoded within these models may later become retracted, disproven, or updated by subsequent research, creating the risk of disseminating outdated information in scientific workflows. This creates a need for LMs to forget obsolete scientific claims. Machine unlearning offers a promising solution by enabling knowledge removal while maintaining overall model utility. Existing studies primarily investigate instance-level forgetting; however, scientific claims introduce additional challenges because they are interconnected, and continually evolving. To address this gap, we introduce the task of Scientific Claim Unlearning and present a new benchmark, SciUnlearn. We show that current unlearning approaches are unable to effectively eliminate claim-level knowledge and often achieve only superficial suppression, highlighting the need for specialized methods designed for structured knowledge removal.

Related papers

• Can LLMs Discover Scientific Laws in Real and Parallel Worlds? [58.76147897928744]
  We introduce SCILAWS-BENCH, a benchmark for scientific law discovery built from published research and real scientific data.<n>It comprises 118 problems drawn from 381 scientific papers, covering 291 candidate laws and roughly 8M real data points across six scientific disciplines.<n>We find that predictive fit can diverge from scientific validity, memorization shapes whether models reproduce or move beyond published formulas, and our best-of-N study reveals a selection bottleneck.
  arXiv  Detail & Related papers  (2026-09-01T17:17:59Z)
• NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents [65.85967483058705]
  Large language models are emerging as powerful tools for scientific law discovery.<n>Existing benchmarks for this task suffer from a fundamental methodological trilemma.<n>We introduce NewtonBench, a benchmark comprising 324 scientific law discovery tasks across 12 physics domains.
  arXiv  Detail & Related papers  (2025-10-08T16:12:11Z)
• The Need for Verification in AI-Driven Scientific Discovery [9.887965168376311]
  Machine learning and large language models can generate hypotheses at a scale and speed far exceeding traditional methods.<n>We argue that without scalable and reliable mechanisms for verification, scientific progress risks being hindered rather than being advanced.
  arXiv  Detail & Related papers  (2025-09-01T11:50:04Z)
• Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning [53.82037883518254]
  We introduce SciReas, a diverse suite of existing benchmarks for scientific reasoning tasks.<n>We then propose KRUX, a probing framework for studying the distinct roles of reasoning and knowledge in scientific tasks.
  arXiv  Detail & Related papers  (2025-08-26T17:04:23Z)
• ScienceMeter: Tracking Scientific Knowledge Updates in Language Models [79.33626657942169]
  Large Language Models (LLMs) are increasingly used to support scientific research, but their knowledge of scientific advancements can quickly become outdated.<n>We introduce ScienceMeter, a new framework for evaluating scientific knowledge update methods over scientific knowledge spanning the past, present, and future.
  arXiv  Detail & Related papers  (2025-05-30T07:28:20Z)
• From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models [3.861418246476623]
  In pursuit of artificial general intelligence (AGI), there is a growing need for models that learn, reason, and generate new knowledge.<n>This survey offers a structured lens to examine Large Language Models-based hypothesis discovery.<n>We synthesize existing work in hypothesis generation, application, and validation, identifying both key achievements and critical gaps.
  arXiv  Detail & Related papers  (2025-05-28T03:40:02Z)
• Advancing the Scientific Method with Large Language Models: From Hypothesis to Discovery [35.888956949646]
  Large Language Models (LLMs) are transforming scientific research by reshaping the scientific method.<n>LLMs are involved in experimental design, data analysis, and enhancing productivity, particularly in chemistry and biology.<n>Transition to AI-driven science raises ethical questions about creativity, oversight, and responsibility.
  arXiv  Detail & Related papers  (2025-05-22T10:05:48Z)
• Decoding complexity: how machine learning is redefining scientific discovery [16.132517461279487]
  Machine learning (ML) has become an essential tool for organising, analysing, and interpreting complex datasets.<n>This paper explores the transformative role of ML in accelerating breakthroughs across a range of scientific disciplines.
  arXiv  Detail & Related papers  (2024-05-07T09:58:02Z)
• SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models [57.96527452844273]
  We introduce SciInstruct, a suite of scientific instructions for training scientific language models capable of college-level scientific reasoning. We curated a diverse and high-quality dataset encompassing physics, chemistry, math, and formal proofs. To verify the effectiveness of SciInstruct, we fine-tuned different language models with SciInstruct, i.e., ChatGLM3 (6B and 32B), Llama3-8B-Instruct, and Mistral-7B: MetaMath.
  arXiv  Detail & Related papers  (2024-01-15T20:22:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.