From Words to Worth: Newborn Article Impact Prediction with LLM

• URL: http://arxiv.org/abs/2408.03934v1
• Date: Wed, 7 Aug 2024 17:52:02 GMT
• Title: From Words to Worth: Newborn Article Impact Prediction with LLM
• Authors: Penghai Zhao, Qinghua Xing, Kairan Dou, Jinyu Tian, Ying Tai, Jian Yang, Ming-Ming Cheng, Xiang Li,
• Abstract summary: This paper introduces a promising approach, leveraging the capabilities of fine-tuned LLMs to predict the future impact of newborn articles. A comprehensive dataset has been constructed and released for fine-tuning the LLM, containing over 12,000 entries with corresponding titles, abstracts, and TNCSI_SP.
• Score: 69.41680520058418
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As the academic landscape expands, the challenge of efficiently identifying potentially high-impact articles among the vast number of newly published works becomes critical. This paper introduces a promising approach, leveraging the capabilities of fine-tuned LLMs to predict the future impact of newborn articles solely based on titles and abstracts. Moving beyond traditional methods heavily reliant on external information, the proposed method discerns the shared semantic features of highly impactful papers from a large collection of title-abstract and potential impact pairs. These semantic features are further utilized to regress an improved metric, TNCSI_SP, which has been endowed with value, field, and time normalization properties. Additionally, a comprehensive dataset has been constructed and released for fine-tuning the LLM, containing over 12,000 entries with corresponding titles, abstracts, and TNCSI_SP. The quantitative results, with an NDCG@20 of 0.901, demonstrate that the proposed approach achieves state-of-the-art performance in predicting the impact of newborn articles when compared to competitive counterparts. Finally, we demonstrate a real-world application for predicting the impact of newborn journal articles to demonstrate its noteworthy practical value. Overall, our findings challenge existing paradigms and propose a shift towards a more content-focused prediction of academic impact, offering new insights for assessing newborn article impact.

Related papers

• ATLAS: Improving Lay Summarisation with Attribute-based Control [19.62666787748948]
  Lay summarisation aims to produce summaries that are comprehensible to non-expert audiences. Previous work assumes a one-size-fits-all approach, where the content and style of the produced summary are entirely dependent on the data used to train the model. We propose ATLAS, a novel abstractive summarisation approach that can control various properties that contribute to the overall "layness" of the generated summary.
  arXiv  Detail & Related papers  (2024-06-09T03:22:55Z)
• Entangled Relations: Leveraging NLI and Meta-analysis to Enhance Biomedical Relation Extraction [35.320291731292286]
  We introduce MetaEntail-RE, a novel adaptation method that harnesses NLI principles to enhance relation extraction. Our approach follows past works by verbalizing relation classes into class-indicative hypotheses. Our experimental results underscore the versatility of MetaEntail-RE, demonstrating performance gains across both biomedical and general domains.
  arXiv  Detail & Related papers  (2024-05-31T23:05:04Z)
• LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition [67.96794382040547]
  $LLM-DA$ is a novel data augmentation technique based on large language models (LLMs) for the few-shot NER task. Our approach involves employing 14 contextual rewriting strategies, designing entity replacements of the same type, and incorporating noise injection to enhance robustness.
  arXiv  Detail & Related papers  (2024-02-22T14:19:56Z)
• P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models [57.571395694391654]
  We find that existing approaches alter the political opinions and stances of news articles in more than 50% of summaries. We propose P3SUM, a diffusion model-based summarization approach controlled by political perspective classifiers. Experiments on three news summarization datasets demonstrate that P3SUM outperforms state-of-the-art summarization systems.
  arXiv  Detail & Related papers  (2023-11-16T10:14:28Z)
• Enhancing Topic Extraction in Recommender Systems with Entropy Regularization [2.7286395031146062]
  This paper introduces a novel approach called entropy regularization to address the issue of low explainability of recommender systems. Experiment results show a significant improvement in topic coherence, which is quantified by cosine similarity on word embeddings.
  arXiv  Detail & Related papers  (2023-06-12T20:05:09Z)
• MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective [57.19660234992812]
  NER model has achieved promising performance on standard NER benchmarks. Recent studies show that previous approaches may over-rely on entity mention information, resulting in poor performance on out-of-vocabulary (OOV) entity recognition. We propose MINER, a novel NER learning framework, to remedy this issue from an information-theoretic perspective.
  arXiv  Detail & Related papers  (2022-04-09T05:18:20Z)
• Deep forecasting of translational impact in medical research [1.8130872753848115]
  We develop a suite of representational and discriminative mathematical models of multi-scale publication data. We show that citations are only moderately predictive of translational impact as judged by inclusion in patents, guidelines, or policy documents. We argue that content-based models of impact are superior in performance to conventional, citation-based measures.
  arXiv  Detail & Related papers  (2021-10-17T19:29:41Z)
• Simplifying Impact Prediction for Scientific Articles [1.8352113484137624]
  Estimating the expected impact of an article is valuable for various applications. We propose a model that can be trained using minimal article metadata.
  arXiv  Detail & Related papers  (2020-12-30T15:24:55Z)
• Reliable Evaluations for Natural Language Inference based on a Unified Cross-dataset Benchmark [54.782397511033345]
  Crowd-sourced Natural Language Inference (NLI) datasets may suffer from significant biases like annotation artifacts. We present a new unified cross-datasets benchmark with 14 NLI datasets and re-evaluate 9 widely-used neural network-based NLI models. Our proposed evaluation scheme and experimental baselines could provide a basis to inspire future reliable NLI research.
  arXiv  Detail & Related papers  (2020-10-15T11:50:12Z)
• Latent Opinions Transfer Network for Target-Oriented Opinion Words Extraction [63.70885228396077]
  We propose a novel model to transfer opinions knowledge from resource-rich review sentiment classification datasets to low-resource task TOWE. Our model achieves better performance compared to other state-of-the-art methods and significantly outperforms the base model without transferring opinions knowledge.
  arXiv  Detail & Related papers  (2020-01-07T11:50:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.