Measuring publication relatedness using controlled vocabularies

• URL: http://arxiv.org/abs/2408.15004v1
• Date: Tue, 27 Aug 2024 12:41:37 GMT
• Title: Measuring publication relatedness using controlled vocabularies
• Authors: Emil Dolmer Alnor,
• Abstract summary: Controlled vocabularies provide a promising basis for measuring relatedness. There exists no comprehensive and direct test of their accuracy and suitability for different types of research questions. This paper reviews existing measures, develops a new measure, and benchmarks the measures using TREC Genomics data as a ground truth of topics.
• Score: 0.0
• License:
• Abstract: Measuring the relatedness between scientific publications has important applications in many areas of bibliometrics and science policy. Controlled vocabularies provide a promising basis for measuring relatedness because they address issues that arise when using citation or textual similarity to measure relatedness. While several controlled-vocabulary-based relatedness measures have been developed, there exists no comprehensive and direct test of their accuracy and suitability for different types of research questions. This paper reviews existing measures, develops a new measure, and benchmarks the measures using TREC Genomics data as a ground truth of topics. The benchmark test show that the new measure and the measure proposed by Ahlgren et al. (2020) have differing strengths and weaknesses. These results inform a discussion of which method to choose when studying interdisciplinarity, information retrieval, clustering of science, and researcher topic switching.

Related papers

• A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why? [84.46288849132634]
  We propose a systematic framework for analyzing the evolution of research topics in a scientific field using causal discovery and inference techniques. We define three variables to encompass diverse facets of the evolution of research topics within NLP. We utilize a causal discovery algorithm to unveil the causal connections among these variables using observational data.
  arXiv  Detail & Related papers  (2023-05-22T11:08:00Z)
• Evaluating generative audio systems and their metrics [80.97828572629093]
  This paper investigates state-of-the-art approaches side-by-side with (i) a set of previously proposed objective metrics for audio reconstruction, and (ii) a listening study. Results indicate that currently used objective metrics are insufficient to describe the perceptual quality of current systems.
  arXiv  Detail & Related papers  (2022-08-31T21:48:34Z)
• SciTweets -- A Dataset and Annotation Framework for Detecting Scientific Online Discourse [2.3371548697609303]
  Scientific topics, claims and resources are increasingly debated as part of online discourse. This has led to both significant societal impact and increased interest in scientific online discourse from various disciplines. Research across disciplines currently suffers from a lack of robust definitions of the various forms of science-relatedness.
  arXiv  Detail & Related papers  (2022-06-15T08:14:55Z)
• The Values and Limits of Altmetrics [0.0]
  Some stakeholders in higher education have championed altmetrics as a new way to understand research impact. This chapter explores the values and limits of altmetrics, including their role in evaluating, promoting, and disseminating research.
  arXiv  Detail & Related papers  (2021-12-17T15:21:35Z)
• Quantitative Evaluations on Saliency Methods: An Experimental Study [6.290238942982972]
  We briefly summarize the status quo of the metrics, including faithfulness, localization, false-positives, sensitivity check, and stability. We conclude that among all the methods we compare, no single explanation method dominates others in all metrics.
  arXiv  Detail & Related papers  (2020-12-31T14:13:30Z)
• GO FIGURE: A Meta Evaluation of Factuality in Summarization [131.1087461486504]
  We introduce GO FIGURE, a meta-evaluation framework for evaluating factuality evaluation metrics. Our benchmark analysis on ten factuality metrics reveals that our framework provides a robust and efficient evaluation. It also reveals that while QA metrics generally improve over standard metrics that measure factuality across domains, performance is highly dependent on the way in which questions are generated.
  arXiv  Detail & Related papers  (2020-10-24T08:30:20Z)
• Domain Divergences: a Survey and Empirical Analysis [47.535524183965464]
  We develop a taxonomy of divergence measures consisting of three classes -- Information-theoretic, Geometric, and Higher-order measures. We perform correlation analysis spanning 130 domain adaptation scenarios, 3 varied NLP tasks and 12 divergence measures identified from our survey.
  arXiv  Detail & Related papers  (2020-10-23T07:12:52Z)
• Uncertainty over Uncertainty: Investigating the Assumptions, Annotations, and Text Measurements of Economic Policy Uncertainty [12.787262921924953]
  We examine an economic index that measures economic policy uncertainty from keyword occurrences in news. We find some annotator disagreements of economic policy uncertainty can be attributed to ambiguity in language.
  arXiv  Detail & Related papers  (2020-10-09T17:50:29Z)
• A Survey on Text Classification: From Shallow to Deep Learning [83.47804123133719]
  The last decade has seen a surge of research in this area due to the unprecedented success of deep learning. This paper fills the gap by reviewing the state-of-the-art approaches from 1961 to 2021. We create a taxonomy for text classification according to the text involved and the models used for feature extraction and classification.
  arXiv  Detail & Related papers  (2020-08-02T00:09:03Z)
• A Survey on Causal Inference [64.45536158710014]
  Causal inference is a critical research topic across many domains, such as statistics, computer science, education, public policy and economics. Various causal effect estimation methods for observational data have sprung up.
  arXiv  Detail & Related papers  (2020-02-05T21:35:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.