From Words to Worth: Newborn Article Impact Prediction with LLM

• URL: http://arxiv.org/abs/2408.03934v2
• Date: Sat, 14 Dec 2024 15:27:41 GMT
• Title: From Words to Worth: Newborn Article Impact Prediction with LLM
• Authors: Penghai Zhao, Qinghua Xing, Kairan Dou, Jinyu Tian, Ying Tai, Jian Yang, Ming-Ming Cheng, Xiang Li,
• Abstract summary: This paper introduces a promising approach, leveraging the capabilities of LLMs to predict the future impact of newborn articles. The proposed method employs LLM to discern the shared semantic features of highly impactful papers from a large collection of title-abstract pairs. The quantitative results, with an MAE of 0.216 and an NDCG@20 of 0.901, demonstrate that the proposed approach achieves state-of-the-art performance.
• Score: 69.41680520058418
• License:
• Abstract: As the academic landscape expands, the challenge of efficiently identifying impactful newly published articles grows increasingly vital. This paper introduces a promising approach, leveraging the capabilities of LLMs to predict the future impact of newborn articles solely based on titles and abstracts. Moving beyond traditional methods heavily reliant on external information, the proposed method employs LLM to discern the shared semantic features of highly impactful papers from a large collection of title-abstract pairs. These semantic features are further utilized to predict the proposed indicator, TNCSI_SP, which incorporates favorable normalization properties across value, field, and time. To facilitate parameter-efficient fine-tuning of the LLM, we have also meticulously curated a dataset containing over 12,000 entries, each annotated with titles, abstracts, and their corresponding TNCSI_SP values. The quantitative results, with an MAE of 0.216 and an NDCG@20 of 0.901, demonstrate that the proposed approach achieves state-of-the-art performance in predicting the impact of newborn articles when compared to several promising methods. Finally, we present a real-world application example for predicting the impact of newborn journal articles to demonstrate its noteworthy practical value. Overall, our findings challenge existing paradigms and propose a shift towards a more content-focused prediction of academic impact, offering new insights for article impact prediction.

Related papers

• Leveraging the Power of LLMs: A Fine-Tuning Approach for High-Quality Aspect-Based Summarization [25.052557735932535]
  Large language models (LLMs) have demonstrated the potential to revolutionize diverse tasks within natural language processing. This paper explores the potential of fine-tuning LLMs for the aspect-based summarization task. We evaluate the impact of fine-tuning open-source foundation LLMs, including Llama2, Mistral, Gemma and Aya, on a publicly available domain-specific aspect based summary dataset.
  arXiv  Detail & Related papers  (2024-08-05T16:00:21Z)
• ATLAS: Improving Lay Summarisation with Attribute-based Control [19.62666787748948]
  Lay summarisation aims to produce summaries that are comprehensible to non-expert audiences. Previous work assumes a one-size-fits-all approach, where the content and style of the produced summary are entirely dependent on the data used to train the model. We propose ATLAS, a novel abstractive summarisation approach that can control various properties that contribute to the overall "layness" of the generated summary.
  arXiv  Detail & Related papers  (2024-06-09T03:22:55Z)
• Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data [102.16105233826917]
  Learning from preference labels plays a crucial role in fine-tuning large language models. There are several distinct approaches for preference fine-tuning, including supervised learning, on-policy reinforcement learning (RL), and contrastive learning.
  arXiv  Detail & Related papers  (2024-04-22T17:20:18Z)
• Debiasing Multimodal Large Language Models [61.6896704217147]
  Large Vision-Language Models (LVLMs) have become indispensable tools in computer vision and natural language processing. Our investigation reveals a noteworthy bias in the generated content, where the output is primarily influenced by the underlying Large Language Models (LLMs) prior to the input image. To rectify these biases and redirect the model's focus toward vision information, we introduce two simple, training-free strategies.
  arXiv  Detail & Related papers  (2024-03-08T12:35:07Z)
• LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition [67.96794382040547]
  $LLM-DA$ is a novel data augmentation technique based on large language models (LLMs) for the few-shot NER task. Our approach involves employing 14 contextual rewriting strategies, designing entity replacements of the same type, and incorporating noise injection to enhance robustness.
  arXiv  Detail & Related papers  (2024-02-22T14:19:56Z)
• Analysis of Multidomain Abstractive Summarization Using Salience Allocation [2.6880540371111445]
  Season is a model designed to enhance summarization by leveraging salience allocation techniques. This paper employs various evaluation metrics such as ROUGE, METEOR, BERTScore, and MoverScore to evaluate the performance of these models fine-tuned for generating abstractive summaries.
  arXiv  Detail & Related papers  (2024-02-19T08:52:12Z)
• To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis [50.31589712761807]
  Large language models (LLMs) are notoriously token-hungry during pre-training, and high-quality text data on the web is approaching its scaling limit for LLMs. We investigate the consequences of repeating pre-training data, revealing that the model is susceptible to overfitting. Second, we examine the key factors contributing to multi-epoch degradation, finding that significant factors include dataset size, model parameters, and training objectives.
  arXiv  Detail & Related papers  (2023-05-22T17:02:15Z)
• Salience Allocation as Guidance for Abstractive Summarization [61.31826412150143]
  We propose a novel summarization approach with a flexible and reliable salience guidance, namely SEASON (SaliencE Allocation as Guidance for Abstractive SummarizatiON) SEASON utilizes the allocation of salience expectation to guide abstractive summarization and adapts well to articles in different abstractiveness.
  arXiv  Detail & Related papers  (2022-10-22T02:13:44Z)
• Adversarial Gradient Driven Exploration for Deep Click-Through Rate Prediction [39.61776002290324]
  We propose a novel exploration method called textbfAdrial textbfGradientversa Driven textbfExploration (AGE) AGE simulates the gradient updating process, which can approximate the influence of the samples of to-be-explored items for the model. The effectiveness of our approach was demonstrated on an open-access academic dataset.
  arXiv  Detail & Related papers  (2021-12-21T12:13:07Z)
• Simplifying Impact Prediction for Scientific Articles [1.8352113484137624]
  Estimating the expected impact of an article is valuable for various applications. We propose a model that can be trained using minimal article metadata.
  arXiv  Detail & Related papers  (2020-12-30T15:24:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.