CRISP: Characterizing Relative Impact of Scholarly Publications

• URL: http://arxiv.org/abs/2603.26791v1
• Date: Wed, 25 Mar 2026 16:42:30 GMT
• Title: CRISP: Characterizing Relative Impact of Scholarly Publications
• Abstract summary: We propose CRISP, which jointly ranks all cited papers within a citing paper using large language models (LLMs)<n>To mitigate LLMs' positional bias, we rank each list three times in a randomized order and aggregate the impact labels through majority voting.<n>CRISP outperforms a prior state-of-the-art impact classifier by +9.5% accuracy and +8.3% F1 on a dataset of human-annotated citations.
• Score: 70.25915589244961
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Assessing a cited paper's impact is typically done by analyzing its citation context in isolation within the citing paper. While this focuses on the most directly relevant text, it prevents relative comparisons across all the works a paper cites. We propose CRISP, which instead jointly ranks all cited papers within a citing paper using large language models (LLMs). To mitigate LLMs' positional bias, we rank each list three times in a randomized order and aggregate the impact labels through majority voting. This joint approach leverages the full citation context, rather than evaluating citations independently, to more reliably distinguish impactful references. CRISP outperforms a prior state-of-the-art impact classifier by +9.5% accuracy and +8.3% F1 on a dataset of human-annotated citations. CRISP further gains efficiency through fewer LLM calls and performs competitively with an open-source model, enabling scalable, cost-effective citation impact analysis. We release our rankings, impact labels, and codebase to support future research.

Related papers

• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• Enhancing the Robustness of Contextual ASR to Varying Biasing Information Volumes Through Purified Semantic Correlation Joint Modeling [63.755562174967274]
  Cross-attention is affected by variations in biasing information volume.<n>We propose a purified semantic correlation joint modeling (PSC-Joint) approach.<n> PSC-Joint achieves average relative F1 score improvements of up to 21.34% on AISHELL-1 and 28.46% on KeSpeech.
  arXiv  Detail & Related papers  (2025-09-07T03:46:59Z)
• Leveraging GANs for citation intent classification and its impact on citation network analysis [0.0]
  Citations play a fundamental role in the scientific ecosystem, serving as a foundation for tracking the flow of knowledge.<n>In this paper, we adopted a GAN-based method to classify citation intents.<n>We also investigated whether filtering citation intents affects the centrality of papers in citation networks.
  arXiv  Detail & Related papers  (2025-05-27T13:16:09Z)
• The Noisy Path from Source to Citation: Measuring How Scholars Engage with Past Research [20.649638393774048]
  We introduce a computational pipeline to quantify citation fidelity at scale.<n>Using full texts of papers, the pipeline identifies citations in citing papers and the corresponding claims in cited papers.<n>Using a quasi-experiment, we establish the "telephone effect" - when citing papers have low fidelity to the original claim, future papers that cite the citing paper and the original have lower fidelity to the original.
  arXiv  Detail & Related papers  (2025-02-27T22:47:03Z)
• HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction [14.731720495144112]
  We introduce the novel concept of core citation, which identifies the critical references that go beyond superficial mentions. We propose $textbfHLM-Cite, a $textbfH$ybrid $textbfL$anguage $textbfM$odel workflow for citation prediction. We evaluate HLM-Cite across 19 scientific fields, demonstrating a 17.6% performance improvement comparing SOTA methods.
  arXiv  Detail & Related papers  (2024-10-10T10:46:06Z)
• ALiiCE: Evaluating Positional Fine-grained Citation Generation [54.19617927314975]
  We propose ALiiCE, the first automatic evaluation framework for fine-grained citation generation. Our framework first parses the sentence claim into atomic claims via dependency analysis and then calculates citation quality at the atomic claim level. We evaluate the positional fine-grained citation generation performance of several Large Language Models on two long-form QA datasets.
  arXiv  Detail & Related papers  (2024-06-19T09:16:14Z)
• CausalCite: A Causal Formulation of Paper Citations [80.82622421055734]
  CausalCite is a new way to measure the significance of a paper by assessing the causal impact of the paper on its follow-up papers. It is based on a novel causal inference method, TextMatch, which adapts the traditional matching framework to high-dimensional text embeddings. We demonstrate the effectiveness of CausalCite on various criteria, such as high correlation with paper impact as reported by scientific experts.
  arXiv  Detail & Related papers  (2023-11-05T23:09:39Z)
• Enhancing Scientific Papers Summarization with Citation Graph [78.65955304229863]
  We redefine the task of scientific papers summarization by utilizing their citation graph. We construct a novel scientific papers summarization dataset Semantic Scholar Network (SSN) which contains 141K research papers in different domains. Our model can achieve competitive performance when compared with the pretrained models.
  arXiv  Detail & Related papers  (2021-04-07T11:13:35Z)
• ImpactCite: An XLNet-based method for Citation Impact Analysis [4.526582372434088]
  Impact analysis enables us to quantify the quality of the citations. XLNet-based solution ImpactCite achieves a new state-of-the-art performance for both citation intent and sentiment classification. Additional efforts have been performed to come up with CSC-Clean corpus, which is a clean and reliable dataset for citation sentiment classification.
  arXiv  Detail & Related papers  (2020-05-05T08:31:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.