From Words to Worth: Newborn Article Impact Prediction with LLM

• URL: http://arxiv.org/abs/2408.03934v2
• Date: Sat, 14 Dec 2024 15:27:41 GMT
• Title: From Words to Worth: Newborn Article Impact Prediction with LLM
• Authors: Penghai Zhao, Qinghua Xing, Kairan Dou, Jinyu Tian, Ying Tai, Jian Yang, Ming-Ming Cheng, Xiang Li,
• Abstract summary: This paper introduces a promising approach, leveraging the capabilities of LLMs to predict the future impact of newborn articles.<n>The proposed method employs LLM to discern the shared semantic features of highly impactful papers from a large collection of title-abstract pairs.<n>The quantitative results, with an MAE of 0.216 and an NDCG@20 of 0.901, demonstrate that the proposed approach achieves state-of-the-art performance.
• Score: 69.41680520058418
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: As the academic landscape expands, the challenge of efficiently identifying impactful newly published articles grows increasingly vital. This paper introduces a promising approach, leveraging the capabilities of LLMs to predict the future impact of newborn articles solely based on titles and abstracts. Moving beyond traditional methods heavily reliant on external information, the proposed method employs LLM to discern the shared semantic features of highly impactful papers from a large collection of title-abstract pairs. These semantic features are further utilized to predict the proposed indicator, TNCSI_SP, which incorporates favorable normalization properties across value, field, and time. To facilitate parameter-efficient fine-tuning of the LLM, we have also meticulously curated a dataset containing over 12,000 entries, each annotated with titles, abstracts, and their corresponding TNCSI_SP values. The quantitative results, with an MAE of 0.216 and an NDCG@20 of 0.901, demonstrate that the proposed approach achieves state-of-the-art performance in predicting the impact of newborn articles when compared to several promising methods. Finally, we present a real-world application example for predicting the impact of newborn journal articles to demonstrate its noteworthy practical value. Overall, our findings challenge existing paradigms and propose a shift towards a more content-focused prediction of academic impact, offering new insights for article impact prediction.

Related papers

• Comprehensive Manuscript Assessment with Text Summarization Using 69707 articles [10.943765373420135]
  We harness Scopus to curate a significantly comprehensive and large-scale dataset of information from 69707 scientific articles. We propose a deep learning methodology for the impact-based classification tasks, which leverages semantic features extracted from the manuscripts and paper metadata.
  arXiv  Detail & Related papers  (2025-03-26T07:56:15Z)
• Leveraging the Power of LLMs: A Fine-Tuning Approach for High-Quality Aspect-Based Summarization [25.052557735932535]
  Large language models (LLMs) have demonstrated the potential to revolutionize diverse tasks within natural language processing. This paper explores the potential of fine-tuning LLMs for the aspect-based summarization task. We evaluate the impact of fine-tuning open-source foundation LLMs, including Llama2, Mistral, Gemma and Aya, on a publicly available domain-specific aspect based summary dataset.
  arXiv  Detail & Related papers  (2024-08-05T16:00:21Z)
• ATLAS: Improving Lay Summarisation with Attribute-based Control [19.62666787748948]
  Lay summarisation aims to produce summaries that are comprehensible to non-expert audiences. Previous work assumes a one-size-fits-all approach, where the content and style of the produced summary are entirely dependent on the data used to train the model. We propose ATLAS, a novel abstractive summarisation approach that can control various properties that contribute to the overall "layness" of the generated summary.
  arXiv  Detail & Related papers  (2024-06-09T03:22:55Z)
• Entangled Relations: Leveraging NLI and Meta-analysis to Enhance Biomedical Relation Extraction [35.320291731292286]
  We introduce MetaEntail-RE, a novel adaptation method that harnesses NLI principles to enhance relation extraction. Our approach follows past works by verbalizing relation classes into class-indicative hypotheses. Our experimental results underscore the versatility of MetaEntail-RE, demonstrating performance gains across both biomedical and general domains.
  arXiv  Detail & Related papers  (2024-05-31T23:05:04Z)
• Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data [102.16105233826917]
  Learning from preference labels plays a crucial role in fine-tuning large language models. There are several distinct approaches for preference fine-tuning, including supervised learning, on-policy reinforcement learning (RL), and contrastive learning.
  arXiv  Detail & Related papers  (2024-04-22T17:20:18Z)
• Debiasing Multimodal Large Language Models [61.6896704217147]
  Large Vision-Language Models (LVLMs) have become indispensable tools in computer vision and natural language processing. Our investigation reveals a noteworthy bias in the generated content, where the output is primarily influenced by the underlying Large Language Models (LLMs) prior to the input image. To rectify these biases and redirect the model's focus toward vision information, we introduce two simple, training-free strategies.
  arXiv  Detail & Related papers  (2024-03-08T12:35:07Z)
• LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition [67.96794382040547]
  $LLM-DA$ is a novel data augmentation technique based on large language models (LLMs) for the few-shot NER task. Our approach involves employing 14 contextual rewriting strategies, designing entity replacements of the same type, and incorporating noise injection to enhance robustness.
  arXiv  Detail & Related papers  (2024-02-22T14:19:56Z)
• Analysis of Multidomain Abstractive Summarization Using Salience Allocation [2.6880540371111445]
  Season is a model designed to enhance summarization by leveraging salience allocation techniques. This paper employs various evaluation metrics such as ROUGE, METEOR, BERTScore, and MoverScore to evaluate the performance of these models fine-tuned for generating abstractive summaries.
  arXiv  Detail & Related papers  (2024-02-19T08:52:12Z)
• P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models [57.571395694391654]
  We find that existing approaches alter the political opinions and stances of news articles in more than 50% of summaries. We propose P3SUM, a diffusion model-based summarization approach controlled by political perspective classifiers. Experiments on three news summarization datasets demonstrate that P3SUM outperforms state-of-the-art summarization systems.
  arXiv  Detail & Related papers  (2023-11-16T10:14:28Z)
• Enhancing Topic Extraction in Recommender Systems with Entropy Regularization [2.7286395031146062]
  This paper introduces a novel approach called entropy regularization to address the issue of low explainability of recommender systems. Experiment results show a significant improvement in topic coherence, which is quantified by cosine similarity on word embeddings.
  arXiv  Detail & Related papers  (2023-06-12T20:05:09Z)
• To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis [50.31589712761807]
  Large language models (LLMs) are notoriously token-hungry during pre-training, and high-quality text data on the web is approaching its scaling limit for LLMs. We investigate the consequences of repeating pre-training data, revealing that the model is susceptible to overfitting. Second, we examine the key factors contributing to multi-epoch degradation, finding that significant factors include dataset size, model parameters, and training objectives.
  arXiv  Detail & Related papers  (2023-05-22T17:02:15Z)
• Salience Allocation as Guidance for Abstractive Summarization [61.31826412150143]
  We propose a novel summarization approach with a flexible and reliable salience guidance, namely SEASON (SaliencE Allocation as Guidance for Abstractive SummarizatiON) SEASON utilizes the allocation of salience expectation to guide abstractive summarization and adapts well to articles in different abstractiveness.
  arXiv  Detail & Related papers  (2022-10-22T02:13:44Z)
• MINER: Improving Out-of-Vocabulary Named Entity Recognition from an Information Theoretic Perspective [57.19660234992812]
  NER model has achieved promising performance on standard NER benchmarks. Recent studies show that previous approaches may over-rely on entity mention information, resulting in poor performance on out-of-vocabulary (OOV) entity recognition. We propose MINER, a novel NER learning framework, to remedy this issue from an information-theoretic perspective.
  arXiv  Detail & Related papers  (2022-04-09T05:18:20Z)
• Deep forecasting of translational impact in medical research [1.8130872753848115]
  We develop a suite of representational and discriminative mathematical models of multi-scale publication data. We show that citations are only moderately predictive of translational impact as judged by inclusion in patents, guidelines, or policy documents. We argue that content-based models of impact are superior in performance to conventional, citation-based measures.
  arXiv  Detail & Related papers  (2021-10-17T19:29:41Z)
• Simplifying Impact Prediction for Scientific Articles [1.8352113484137624]
  Estimating the expected impact of an article is valuable for various applications. We propose a model that can be trained using minimal article metadata.
  arXiv  Detail & Related papers  (2020-12-30T15:24:55Z)
• Reliable Evaluations for Natural Language Inference based on a Unified Cross-dataset Benchmark [54.782397511033345]
  Crowd-sourced Natural Language Inference (NLI) datasets may suffer from significant biases like annotation artifacts. We present a new unified cross-datasets benchmark with 14 NLI datasets and re-evaluate 9 widely-used neural network-based NLI models. Our proposed evaluation scheme and experimental baselines could provide a basis to inspire future reliable NLI research.
  arXiv  Detail & Related papers  (2020-10-15T11:50:12Z)
• Latent Opinions Transfer Network for Target-Oriented Opinion Words Extraction [63.70885228396077]
  We propose a novel model to transfer opinions knowledge from resource-rich review sentiment classification datasets to low-resource task TOWE. Our model achieves better performance compared to other state-of-the-art methods and significantly outperforms the base model without transferring opinions knowledge.
  arXiv  Detail & Related papers  (2020-01-07T11:50:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.