ClaimFlow: Tracing the Evolution of Scientific Claims in NLP

• URL: http://arxiv.org/abs/2603.16073v1
• Date: Tue, 17 Mar 2026 02:43:36 GMT
• Title: ClaimFlow: Tracing the Evolution of Scientific Claims in NLP
• Abstract summary: We introduce $texttClaimFlow$, a claim-centric view of the NLP literature.<n>We define a new task $-$ $textitClaim Relation Classification$ $-$ which requires models to infer the scientific stance toward a cited claim.<n>Our analysis reveals that $63.5$% claims are never reused; only $11.1$% are ever challenged.
• Score: 67.23189226608389
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Scientific papers do more than report results $-$ they advance $\textit{claims}$ that later work supports, extends, or sometimes refutes. Yet existing methods for citation and claim analysis capture only fragments of this dialogue. In this work, we make these interactions explicit at the level of individual scientific claims. We introduce $\texttt{ClaimFlow}$, a claim-centric view of the NLP literature, built from $304$ ACL Anthology papers (1979$-$2025) that are manually annotated with $1{,}084$ claims and $832$ cross-paper claim relations, indicating whether a citing paper $\textit{supports}$, $\textit{extends}$, $\textit{qualifies}$, $\textit{refutes}$, or references a claim as $\textit{background}$. Using $\texttt{ClaimFlow}$, we define a new task $-$ $\textit{Claim Relation Classification}$ $-$ which requires models to infer the scientific stance toward a cited claim from the text and citation context. Evaluating strong neural models and large language models on this task, we report baseline performance of $0.78$ macro-F1, highlighting that claim-relation classification is feasible but challenging. We further apply our model to $\sim$$13k$ NLP papers to analyze how claims evolve across decades of NLP research. Our analysis reveals that $63.5$% claims are never reused; only $11.1$% are ever challenged; meanwhile, widely propagated claims are more often $\textit{reshaped}$ through qualification and extension than directly confirmed or refuted. Overall, $\texttt{ClaimFlow}$ offers a lens for examining how ideas shift and mature within NLP, and a foundation for assessing whether models can interpret scientific argumentation.

Related papers

• MUSES: A Benchmark for Prospective Intellectual-Roots Retrieval [6.572091819435388]
  textbfMUSES is a million-instance benchmark for prospective intellectual-roots retrieval over a fixed 2.33M-paper corpus.<n>textbfCiteRoots pairs a rhetorical layer over local citation text with a paper-level author-endorsed layer.<n> MUSES organizes difficulty along two axes: a emphfamiliarity axis spanning CiteNext, CiteNew, and CiteNew-Isolated, and a emphfunctional axis spanning broad citations, rhetorical roots, and author-endorsed roots.
  arXiv  Detail & Related papers  (2026-08-31T20:05:36Z)
• An Interactive Agent for Requirement-Driven Candidate Sourcing [75.63902836826038]
  We present sys, the first requirements-driven candidate-sourcing agent.<n>It elicits, validates, retrieves, and verifies a vague people-request into a justified slate.<n>It dominates breadth ($100%$ coverage at $2.5times$ the yield) and is emphnear-orthogonal to the field.
  arXiv  Detail & Related papers  (2026-08-24T17:04:16Z)
• FormalTCS: Benchmarking End-to-End Frontier Formal Theoretical Computer Science Research of Large Language Models [53.68149869349268]
  Large language models (LLMs) have shown growing potential for automated theoretical computer science (TCS) research.<n>We introduce ourbenchmark, an expert-validated benchmark for evaluating LLMs on frontier, end-to-end TCS research.
  arXiv  Detail & Related papers  (2026-08-20T15:13:41Z)
• How Does Research Evolve? Tracing Cross-Domain Trajectories in NLP, ML, and CV with Claim-Grounded Typed Citations [0.0]
  We propose the first claim-grounded typed citation graph in which each edge is linked to the specific claim sentence that motivates it.<n>SciTraj contains 32,559 papers from NLP, ML, and Vision (2015--2024), connected by 573,126 directed edges across six relation types.<n>The corpus also contains 287M typed trajectories of length $geq 3$, covering 72.8% of papers, and supports a temporally split typed link-prediction benchmark.
  arXiv  Detail & Related papers  (2026-06-21T05:20:02Z)
• Subtle Injection for Ground-truth Inference of LLM Training Data [0.2538209532048867]
  SIGIL is a framework that embeds imperceptible emphcanary sequences into protected text and code.<n> SIGIL defines five canary strategies -- lexical-rare, lexical-phrase, syntactic, semantic, and code-pattern.
  arXiv  Detail & Related papers  (2026-05-18T14:48:44Z)
• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• GENIUS: Generative Fluid Intelligence Evaluation Suite [45.98061608718251]
  We introduce $textbfGENIUS$ ($textbfGEN$ Fluid $textbfI$ntelligence Eval$textbfU$ation $textbfS$uite)<n>We formalize $textitGFI$ as a synthesis of three primitives. These include $textitInducing Implicit Patterns$ (e.g., inferring personalized visual preferences), $textitExecuting Ad-hoc Constraints$ (e.g., visualizing abstract metaphors), and
  arXiv  Detail & Related papers  (2026-02-11T18:55:54Z)
• ConvexBench: Can LLMs Recognize Convex Functions? [70.53167848190624]
  Convex analysis is a modern branch of mathematics with many applications.<n>As Large Language Models (LLMs) start to automate research-level math and sciences, it is important for LLMs to demonstrate the ability to understand and reason with convexity.<n>We introduce cb, a scalable and mechanically verifiable benchmark for testing textitwhether LLMs can identify the convexity of a symbolic objective under deep functional composition.
  arXiv  Detail & Related papers  (2026-02-01T07:41:17Z)
• Simple Convergence Proof of Adam From a Sign-like Descent Perspective [58.89890024903816]
  We show that Adam achieves the optimal rate of $cal O(frac1Ts14)$ rather than the previous $cal O(fracln TTs14)$.<n>Our theoretical analysis provides new insights into the role of momentum as a key factor ensuring convergence.
  arXiv  Detail & Related papers  (2025-07-08T13:19:26Z)
• HypER: Literature-grounded Hypothesis Generation and Distillation with Provenance [28.51792655478926]
  Large Language models have demonstrated promising performance in research ideation across scientific domains.<n>We present $textttHypER$ ($textbfHyp$othesis Generation with $textbfE$xplanation and $textbfR$easoning), a small language model trained for literature-guided reasoning and evidence-based hypothesis generation.
  arXiv  Detail & Related papers  (2025-06-15T18:41:23Z)
• Language Models May Verbatim Complete Text They Were Not Explicitly Trained On [97.3414396208613]
  We show that a $n$-gram based membership definition can be effectively gamed.<n>We show that it is difficult to find a single viable choice of $n$ for membership definitions.<n>Our findings highlight the inadequacy of $n$-gram membership, suggesting membership definitions fail to account for auxiliary information.
  arXiv  Detail & Related papers  (2025-03-21T19:57:04Z)
• HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction [14.731720495144112]
  We introduce the novel concept of core citation, which identifies the critical references that go beyond superficial mentions. We propose $textbfHLM-Cite, a $textbfH$ybrid $textbfL$anguage $textbfM$odel workflow for citation prediction. We evaluate HLM-Cite across 19 scientific fields, demonstrating a 17.6% performance improvement comparing SOTA methods.
  arXiv  Detail & Related papers  (2024-10-10T10:46:06Z)
• Inertial Confinement Fusion Forecasting via Large Language Models [48.76222320245404]
  In this study, we introduce $textbfLPI-LLM$, a novel integration of Large Language Models (LLMs) with classical reservoir computing paradigms. We propose the $textitLLM-anchored Reservoir$, augmented with a $textitFusion-specific Prompt$, enabling accurate forecasting of $textttLPI$-generated-hot electron dynamics during implosion. We also present $textbfLPI4AI$, the first $textttLPI$ benchmark based
  arXiv  Detail & Related papers  (2024-07-15T05:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.