Related papers: Unified Examination of Entity Linking in Absence of Candidate Sets

Unified Examination of Entity Linking in Absence of Candidate Sets

URL: http://arxiv.org/abs/2404.11061v1
Date: Wed, 17 Apr 2024 04:37:58 GMT
Title: Unified Examination of Entity Linking in Absence of Candidate Sets
Authors: Nicolas Ong, Hassan Shavarani, Anoop Sarkar,
Abstract summary: We use an ablation study to investigate the impact of candidate sets on the performance of entity linking. We show the trade-off between less restrictive candidate sets, increased inference time and memory footprint for some models.
Score: 3.55026004901472
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Despite remarkable strides made in the development of entity linking systems in recent years, a comprehensive comparative analysis of these systems using a unified framework is notably absent. This paper addresses this oversight by introducing a new black-box benchmark and conducting a comprehensive evaluation of all state-of-the-art entity linking methods. We use an ablation study to investigate the impact of candidate sets on the performance of entity linking. Our findings uncover exactly how much such entity linking systems depend on candidate sets, and how much this limits the general applicability of each system. We present an alternative approach to candidate sets, demonstrating that leveraging the entire in-domain candidate set can serve as a viable substitute for certain models. We show the trade-off between less restrictive candidate sets, increased inference time and memory footprint for some models.

Related papers

Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency [87.16283281290053]
Previous entity disambiguation (ED) methods adopt a discriminative paradigm, where prediction is made based on matching scores between mention context and candidate entities. We propose CoherentED, an ED system equipped with novel designs aimed at enhancing the coherence of entity predictions. We achieve new state-of-the-art results on popular ED benchmarks, with an average improvement of 1.3 F1 points.
arXiv Detail & Related papers (2023-11-06T16:40:13Z)
Summarization from Leaderboards to Practice: Choosing A Representation Backbone and Ensuring Robustness [21.567112955050582]
In both automatic and human evaluation, BART performs better than PEG and T5. We find considerable variation in system output that can be captured only with human evaluation.
arXiv Detail & Related papers (2023-06-18T13:35:41Z)
Better Understanding Differences in Attribution Methods via Systematic Evaluations [57.35035463793008]
Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models.
arXiv Detail & Related papers (2023-03-21T14:24:58Z)
In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation [92.51773744318119]
This paper empirically investigates the strengths and weaknesses of different model selection criteria. We highlight that there is a complex interplay between selection strategies, candidate estimators and the data used for comparing them.
arXiv Detail & Related papers (2023-02-06T16:55:37Z)
Proxy-based Zero-Shot Entity Linking by Effective Candidate Retrieval [3.1498833540989413]
We show that pairing a proxy-based metric learning loss with an adversarial regularizer provides an efficient alternative to hard negative sampling in the candidate retrieval stage. In particular, we show competitive performance on the recall@1 metric, thereby providing the option to leave out the expensive candidate ranking step.
arXiv Detail & Related papers (2023-01-30T22:43:21Z)
Hybrid Rule-Neural Coreference Resolution System based on Actor-Critic Learning [53.73316523766183]
Coreference resolution systems need to tackle two main tasks. One task is to detect all of the potential mentions, the other is to learn the linking of an antecedent for each possible mention. We propose a hybrid rule-neural coreference resolution system based on actor-critic learning.
arXiv Detail & Related papers (2022-12-20T08:55:47Z)
Relational Proxies: Emergent Relationships as Fine-Grained Discriminators [52.17542855760418]
We propose a novel approach that leverages information between the global and local part of an object for encoding its label. We design Proxies based on our theoretical findings and evaluate it on seven challenging fine-grained benchmark datasets. We also experimentally validate our theory and obtain consistent results across multiple benchmarks.
arXiv Detail & Related papers (2022-10-05T11:08:04Z)
Are We There Yet? A Decision Framework for Replacing Term Based Retrieval with Dense Retrieval Systems [35.77217529138364]
Several dense retrieval (DR) models have demonstrated competitive performance to term-based retrieval. DR projects queries and documents into a dense vector space and retrieves results via (approximate) nearest neighbor search. It is impossible to predict whether DR will become ubiquitous in the future, but one way this is possible is through repeated applications of decision processes.
arXiv Detail & Related papers (2022-06-26T23:16:05Z)
UniCon: Unified Context Network for Robust Active Speaker Detection [111.90529347692723]
We introduce a new efficient framework, the Unified Context Network (UniCon), for robust active speaker detection (ASD) Our solution is a novel, unified framework that focuses on jointly modeling multiple types of contextual information. A thorough ablation study is performed on several challenging ASD benchmarks under different settings.
arXiv Detail & Related papers (2021-08-05T13:25:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.