Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

• URL: http://arxiv.org/abs/2609.10745v1
• Date: Wed, 09 Sep 2026 18:41:35 GMT
• Title: Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking
• Abstract summary: Multimodal entity linking grounds entity mentions in text and images to knowledge-base entries.<n>These systems degrade on rare entities, but prior work measures rarity primarily through popularity-based metrics such as pageviews.<n>We broaden this view using knowledge-graph structural metrics that capture how well an entity is documented and connected.<n>Across the resulting rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, showing that different rarity definitions expose different failure modes.
• Score: 53.172843070114475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal entity linking grounds entity mentions in text and images to knowledge-base entries. These systems degrade on rare entities, but prior work measures rarity primarily through popularity-based metrics such as pageviews. We broaden this view using knowledge-graph structural metrics that capture how well an entity is documented and connected. These metrics identify many rare entities that popularity metrics miss. Across the resulting rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, showing that different rarity definitions expose different failure modes. To address these failures, we introduce a simple, training-free framework in which a reasoning-capable vision-language model iteratively searches and reasons over Wikipedia, gathering evidence dynamically. Controlled experiments show that reasoning and retrieval are complementary. Reasoning alone does not significantly improve accuracy on rare entities. Retrieval without reasoning improves rare-entity accuracy but can hurt overall accuracy. Their combination performs best. On MERLIN, a multilingual multimodal entity linking benchmark over five languages (Hindi, Indonesian, Japanese, Tamil, Vietnamese), our best system improves over the state of the art by 6.9% overall and by up to 23.3% on rare-entity slices. We release MERLIN-Rare, rare-entity test slices for targeted evaluation, with our framework.

Related papers

• Lower-Resource, Higher Scores: Language Bias in LLM Evaluators [27.959960538188753]
  We conduct experiments with semantically identical instruction-response pairs across 23 languages.<n>We find that multilingual evaluators assign significantly different scores to different evaluation languages.
  arXiv  Detail & Related papers  (2026-07-16T01:53:09Z)
• Beyond Input Understanding: Diagnosing Multilingual Mathematical Reasoning with Directed Acyclic Trace Graphs [54.72295694649798]
  Large reasoning models (LRMs) achieve strong mathematical reasoning performance in English, but remain much less reliable in many low- and medium-resource languages.<n>We show that controlling the model's reasoning language can substantially reduce accuracy, suggesting that language also affects reasoning execution itself.<n>Experiments on the Qwen3 series across 12 languages show that non-English reasoning often suffers from reduced anchor coverage and weaker dependency fidelity.
  arXiv  Detail & Related papers  (2026-05-26T21:41:52Z)
• TailNLG: A Multilingual Benchmark Addressing Verbalization of Long-Tail Entities [35.1254326443212]
  We present the first systematic study of long-tail entities in Data-to-Text generation.<n>We introduce TailNLG, a new multilingual benchmark in English, Italian, and Spanish, built from Wikidata.<n>We show that the impact of long-tail entities varies across models and languages, and that existing evaluation metrics do not consistently capture these differences.
  arXiv  Detail & Related papers  (2026-03-29T17:01:54Z)
• CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data [56.043078390377076]
  We introduce CommonLID, a community-driven, human-annotated LID benchmark for the web domain.<n>We show CommonLID's value by using it, alongside five other common evaluation sets, to test eight popular LID models.<n>We highlight that existing evaluations overestimate LID accuracy for many languages in the web domain.
  arXiv  Detail & Related papers  (2026-01-25T22:49:30Z)
• Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation [57.11989521509119]
  We propose a novel agentic translation evaluation framework, centered by a reflective Core Agent that invokes specialized sub-agents.<n> Experimental results indicate the efficacy of RATE, achieving an improvement of at least 3.2 meta score compared with current metrics.
  arXiv  Detail & Related papers  (2026-01-12T09:03:42Z)
• Detecting Corpus-Level Knowledge Inconsistencies in Wikipedia with Large Language Models [11.16952630564181]
  We focus on inconsistencies, a specific type of factual inaccuracy, and introduce the task of corpus-level inconsistency detection.<n>We present CLAIRE, an agentic system that combines LLM reasoning with retrieval to surface potentially inconsistent claims along with contextual evidence for human review.<n>In a user study with experienced Wikipedia editors, 87.5% reported higher confidence when using CLAIRE, and participants identified 64.7% more inconsistencies in the same amount of time.
  arXiv  Detail & Related papers  (2025-09-27T10:32:41Z)
• Cobra Effect in Reference-Free Image Captioning Metrics [58.438648377314436]
  A proliferation of reference-free methods, leveraging visual-language pre-trained models (VLMs), has emerged. In this paper, we study if there are any deficiencies in reference-free metrics. We employ GPT-4V as an evaluative tool to assess generated sentences and the result reveals that our approach achieves state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2024-02-18T12:36:23Z)
• Detecting and Mitigating Hallucinations in Multilingual Summarisation [40.5267502712576]
  Hallucinations pose a significant challenge to the reliability of neural models for abstractive summarisation. We develop a novel metric, mFACT, evaluating the faithfulness of non-English summaries. We then propose a simple but effective method to reduce hallucinations with a cross-lingual transfer.
  arXiv  Detail & Related papers  (2023-05-23T02:59:25Z)
• Entity Disambiguation with Entity Definitions [50.01142092276296]
  Local models have recently attained astounding performances in Entity Disambiguation (ED) Previous works limited their studies to using, as the textual representation of each candidate, only its Wikipedia title. In this paper, we address this limitation and investigate to what extent more expressive textual representations can mitigate it. We report a new state of the art on 2 out of 6 benchmarks we consider and strongly improve the generalization capability over unseen patterns.
  arXiv  Detail & Related papers  (2022-10-11T17:46:28Z)
• Reliable Fidelity and Diversity Metrics for Generative Models [30.941563781926202]
  The most widely used metric for measuring the similarity between real and generated images has been the Fr'echet Inception Distance (FID) score. We show that even the latest version of the precision and recall metrics are not reliable yet. We propose density and coverage metrics that solve the above issues.
  arXiv  Detail & Related papers  (2020-02-23T00:50:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.