Claim2Source at CheckThat! 2026: Improving Multilingual Scientific Claim-Source Retrieval with Verification-based Re-Ranking
- URL: http://arxiv.org/abs/2607.04043v1
- Date: Sat, 04 Jul 2026 22:07:23 GMT
- Title: Claim2Source at CheckThat! 2026: Improving Multilingual Scientific Claim-Source Retrieval with Verification-based Re-Ranking
- Abstract summary: We propose a multi-stage retrieval framework for multilingual scientific claim-source retrieval.<n>The framework employs bilingual claim representations, metadata-enhanced source representations, and language-specific adaptation of dense retrieval models.<n>Our approach achieves an average MRR@5 score of 0.7628 across English, German, and French claims, ranking first on the CheckThat! 2026 leaderboard.
- Score: 17.04255262988263
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Multilingual scientific claim-source retrieval aims to identify the scientific publication supporting a claim shared on social media. This task is challenging because claims often differ from source publications in terms of language, wording, and level of detail, which weakens the connection between claims and their underlying evidence. In this paper, we present our approach for the CheckThat! 2026 Lab Task 1: Source Retrieval for Scientific Web Claims. We propose a multi-stage retrieval framework for multilingual scientific claim-source retrieval that combines structured claim and source representations with progressive candidate refinement. To address multilingual retrieval challenges, the framework employs bilingual claim representations, metadata-enhanced source representations, and language-specific adaptation of dense retrieval models. Building on this setup, a first-stage retriever generates an initial pool of candidate sources, after which similarity-based re-ranking improves the ranking of highly relevant sources and verification-based re-ranking identifies the candidate source that best supports the claim using verification signals. Our approach achieves an average MRR@5 score of 0.7628 across English, German, and French claims, ranking first on the CheckThat! 2026 leaderboard.
Related papers
- Scientific Claim-Source Retrieval Revisited: A Comparative Study of Style Transfer and Re-Ranking [17.04255262988263]
We present a comparative study of scientific claim-source retrieval on the CheckThat! 2026 benchmark.<n>Our results show that translating claims into English outperforms both original and bilingual claim representations.<n>We introduce three novel re-ranking models based on attribution, entity overlap, and verification-based reasoning.
arXiv Detail & Related papers (2026-07-17T11:43:37Z) - Beyond Monolingual Deep Research: Evaluating Agents and Retrievers with Cross-Lingual BrowseComp-Plus [35.39357302837359]
XBCP (Cross-lingual BrowseComp-Plus) is a controlled benchmark that preserves the English question-and-answer space of BrowseComp-Plus but varies the languages of the supporting documents.<n>XBCP instantiates two complementary settings: in the cross-lingual setting, each query is paired with evidence in a single assigned language.<n>We evaluate four deep research agents using sparse and dense multilingual retrievers, measuring answer accuracy, evidence recall, search behavior, calibration, citation fidelity, and oracle retrieval.
arXiv Detail & Related papers (2026-06-13T15:11:52Z) - The Multilingual Curse at the Retrieval Layer: Evidence from Amharic [49.25758237235551]
Strong zero-shot scores on multilingual benchmarks are often taken as evidence that current encoders transfer reliably across many languages.<n>We argue that this assumption breaks down for underrepresented, morphologically rich languages.<n>We compare zero-shot multilingual retrievers, Amharic-fine-tuned multilingual retrievers, and monolingual Amharic retrievers.
arXiv Detail & Related papers (2026-05-23T12:44:30Z) - MeVer at CheckThat! 2026: Cluster-Aware Hard-Negative Mining for Multilingual Scientific-Source Retrieval [11.097006771680896]
We present our submission to CheckThat! 2026 Task 1 on multilingual scientific-source retrieval.<n>We propose cluster-aware hard-negative mining strategies that exploit the semantic structure of retrieved candidate pools.<n>Our experiments show that different hard-negative structures induce different retrieval behaviors.
arXiv Detail & Related papers (2026-05-22T21:24:14Z) - Retrieval or Representation? Reassessing Benchmark Gaps in Multilingual and Visually Rich RAG [1.4425299138308667]
BM25 rank documents by term overlap with corpus-level weighting.<n>End-to-end multimodal retrievers trained on large query-document datasets claim substantial improvements over these approaches.<n>We demonstrate that better document representation is the primary driver of benchmark improvements.
arXiv Detail & Related papers (2026-03-04T16:21:20Z) - NeuCLIRTech: Chinese Monolingual and Cross-Language Information Retrieval Evaluation in a Challenging Domain [49.3943974580576]
This paper presents NeuCLIRTech, an evaluation collection for cross-language retrieval over technical information.<n>The collection consists of technical documents written in Chinese and those same documents machine translated into English.<n>The collection supports two retrieval scenarios: monolingual retrieval in Chinese, and cross-language retrieval with English as the query language.
arXiv Detail & Related papers (2026-02-05T05:57:55Z) - Investigating Language and Retrieval Bias in Multilingual Previously Fact-Checked Claim Detection [4.6738956348193]
Large Language Models (LLMs) offer powerful capabilities for cross-lingual fact-checking.<n>LLMs often exhibit language bias, performing disproportionately better on high-resource languages such as English.<n>We present and inspect a novel concept - retrieval bias, when information retrieval systems tend to favor certain information over others.
arXiv Detail & Related papers (2025-09-29T17:50:32Z) - Query Expansion Using Contextual Clue Sampling with Language Models [69.51976926838232]
We propose a combination of an effective filtering strategy and fusion of the retrieved documents based on the generation probability of each context.
Our lexical matching based approach achieves a similar top-5/top-20 retrieval accuracy and higher top-100 accuracy compared with the well-established dense retrieval model DPR.
For end-to-end QA, the reader model also benefits from our method and achieves the highest Exact-Match score against several competitive baselines.
arXiv Detail & Related papers (2022-10-13T15:18:04Z) - CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual
Retrieval [73.48591773882052]
Most fact-checking approaches focus on English only due to the data scarcity issue in other languages.
We present the first fact-checking framework augmented with crosslingual retrieval.
We train the retriever with our proposed Crosslingual Inverse Cloze Task (XICT)
arXiv Detail & Related papers (2022-09-05T17:36:14Z) - On Cross-Lingual Retrieval with Multilingual Text Encoders [51.60862829942932]
We study the suitability of state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks.
We benchmark their performance in unsupervised ad-hoc sentence- and document-level CLIR experiments.
We evaluate multilingual encoders fine-tuned in a supervised fashion (i.e., we learn to rank) on English relevance data in a series of zero-shot language and domain transfer CLIR experiments.
arXiv Detail & Related papers (2021-12-21T08:10:27Z) - Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval [51.004601358498135]
Mr. TyDi is a benchmark dataset for mono-lingual retrieval in eleven typologically diverse languages.
The goal of this resource is to spur research in dense retrieval techniques in non-English languages.
arXiv Detail & Related papers (2021-08-19T16:53:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.