Predicting Award Winning Research Papers at Publication Time

• URL: http://arxiv.org/abs/2406.12535v1
• Date: Tue, 18 Jun 2024 12:09:10 GMT
• Title: Predicting Award Winning Research Papers at Publication Time
• Authors: Riccardo Vella, Andrea Vitaletti, Fabrizio Silvestri,
• Abstract summary: We predict the likelihood for a research paper of winning an award only relying on information available at publication time. We made our experiments considering the ArnetMiner citation graph, while the ground truth on award-winning papers has been obtained from a collection of best paper awards from 32 computer science conferences. Remarkably, The high recall and the low false negatives rate, show how the model performs very well at identifying papers that will not win an award.
• Score: 5.318302558039615
• License:
• Abstract: In recent years, many studies have been focusing on predicting the scientific impact of research papers. Most of these predictions are based on citations count or rely on features obtainable only from already published papers. In this study, we predict the likelihood for a research paper of winning an award only relying on information available at publication time. For each paper, we build the citation subgraph induced from its bibliography. We initially consider some features of this subgraph, such as the density and the global clustering coefficient, to make our prediction. Then, we mix this information with textual features, extracted from the abstract and the title, to obtain a more accurate final prediction. We made our experiments considering the ArnetMiner citation graph, while the ground truth on award-winning papers has been obtained from a collection of best paper awards from 32 computer science conferences. In our experiment, we obtained an encouraging F1 score of 0.694. Remarkably, The high recall and the low false negatives rate, show how the model performs very well at identifying papers that will not win an award. This behavior can help researchers in getting a first evaluation of their work at publication time. Lastly, we made some first experiments on interpretability. Our results highlight some interesting patterns both in topological and textual features.

Related papers

• CausalCite: A Causal Formulation of Paper Citations [80.82622421055734]
  CausalCite is a new way to measure the significance of a paper by assessing the causal impact of the paper on its follow-up papers. It is based on a novel causal inference method, TextMatch, which adapts the traditional matching framework to high-dimensional text embeddings. We demonstrate the effectiveness of CausalCite on various criteria, such as high correlation with paper impact as reported by scientific experts.
  arXiv  Detail & Related papers  (2023-11-05T23:09:39Z)
• Estimating the Causal Effect of Early ArXiving on Paper Acceptance [56.538813945721685]
  We estimate the effect of arXiving a paper before the reviewing period (early arXiving) on its acceptance to the conference. Our results suggest that early arXiving may have a small effect on a paper's chances of acceptance.
  arXiv  Detail & Related papers  (2023-06-24T07:45:38Z)
• Cracking Double-Blind Review: Authorship Attribution with Deep Learning [43.483063713471935]
  We propose a transformer-based, neural-network architecture to attribute an anonymous manuscript to an author. We leverage all research papers publicly available on arXiv amounting to over 2 million manuscripts. Our method achieves an unprecedented authorship attribution accuracy, where up to 73% of papers are attributed correctly.
  arXiv  Detail & Related papers  (2022-11-14T15:50:24Z)
• Citation Trajectory Prediction via Publication Influence Representation Using Temporal Knowledge Graph [52.07771598974385]
  Existing approaches mainly rely on mining temporal and graph data from academic articles. Our framework is composed of three modules: difference-preserved graph embedding, fine-grained influence representation, and learning-based trajectory calculation. Experiments are conducted on both the APS academic dataset and our contributed AIPatent dataset.
  arXiv  Detail & Related papers  (2022-10-02T07:43:26Z)
• Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold [88.83876819883653]
  We show through a manual classification of recent NLP research papers that this is indeed the case. We observe that NLP research often goes beyond the square one setup, focusing not only on accuracy, but also on fairness or interpretability, but typically only along a single dimension.
  arXiv  Detail & Related papers  (2022-06-20T13:04:23Z)
• A Measure of Research Taste [91.3755431537592]
  We present a citation-based measure that rewards both productivity and taste. The presented measure, CAP, balances the impact of publications and their quantity. We analyze the characteristics of CAP for highly-cited researchers in biology, computer science, economics, and physics.
  arXiv  Detail & Related papers  (2021-05-17T18:01:47Z)
• Enhancing Scientific Papers Summarization with Citation Graph [78.65955304229863]
  We redefine the task of scientific papers summarization by utilizing their citation graph. We construct a novel scientific papers summarization dataset Semantic Scholar Network (SSN) which contains 141K research papers in different domains. Our model can achieve competitive performance when compared with the pretrained models.
  arXiv  Detail & Related papers  (2021-04-07T11:13:35Z)
• Early Indicators of Scientific Impact: Predicting Citations with Altmetrics [0.0]
  We use altmetrics to predict the short-term and long-term citations that a scholarly publication could receive. We build various classification and regression models and evaluate their performance, finding neural networks and ensemble models to perform best for these tasks.
  arXiv  Detail & Related papers  (2020-12-25T16:25:07Z)
• Longitudinal Citation Prediction using Temporal Graph Neural Networks [27.589741169713825]
  We introduce the task of sequence citation prediction. The goal is to accurately predict the trajectory of the number of citations a scholarly work receives over time.
  arXiv  Detail & Related papers  (2020-12-10T15:25:16Z)
• Utilizing Citation Network Structure to Predict Citation Counts: A Deep Learning Approach [0.0]
  This paper proposes an end-to-end deep learning network, DeepCCP, which combines the effect of information cascade and looks at the citation counts prediction problem. According to experiments on 6 real data sets, DeepCCP is superior to the state-of-the-art methods in terms of the accuracy of citation count prediction.
  arXiv  Detail & Related papers  (2020-09-06T05:27:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.