SciClaimSeekers at CheckThat! 2026: Retrieving Scientific Sources for Social Media Claims with LLM Reranking

• URL: http://arxiv.org/abs/2607.24803v1
• Date: Mon, 06 Jul 2026 16:36:46 GMT
• Title: SciClaimSeekers at CheckThat! 2026: Retrieving Scientific Sources for Social Media Claims with LLM Reranking
• Abstract summary: SciClaimSeekers is a retrieval and reranking framework.<n>System combines BM25 and zero-shot multilingual E5 retrieval with Reciprocal Rank Fusion.<n>Our experiment suggests that large pre-trained models, when combined into a careful pipeline, can be competitive on this task.
• Score: 3.969046654861534
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scientific claims often spread on social media faster than they can be verified, while posts rarely link to the original scholarly sources. To tackle this problem this paper presents system called SciClaimSeekers, a retrieval and reranking framework by combining BM25 and zero-shot multilingual E5 retrieval with Reciprocal Rank Fusion (k=60), followed by Qwen2.5-14B-Instruct pointwise reranking. The pipeline reaches 64.36% MRR@5 on the English development set a 13.67-point jump over BM25 and 10.17 points over the unranked hybrid and 64.39% on the official test set, in the CLEF-2026 CheckThat! Task 1 evaluation. Our experiment suggests that large pre-trained models, when combined into a careful pipeline, can be competitive with fine-tuned approaches on this task.

Related papers

• Comparing Retrieval Methods for Academic Advisor Discovery: A Six-Method Study of 768 CS Faculty Profiles Across 9 US Universities [0.0]
  Methods span sparse lexical matching, TF-IDF, BM25, dense semantic retrieval, hybrid score fusion, and learning-to-rank.<n>We present a comparative evaluation of six information retrieval methods for the task of academic advisor discovery.
  arXiv  Detail & Related papers  (2026-09-03T14:19:14Z)
• PaperGym: Rubric-Centered Evolution for Research-Plan Generation [69.5921878728483]
  We introduce PaperGym, a unified framework that turns each research paper into a complete training environment.<n>PaperGym exploits the structure of a paper: the question is synthesized from the research goal and background, while the criteria are derived from the method and experiments.<n>With the recipe held fixed, models trained on PaperGym-20k win 58.1% of three-way comparisons, against 28.2% forHub Science.
  arXiv  Detail & Related papers  (2026-08-31T17:31:18Z)
• From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop [63.00693129203923]
  The Workshop on Trustworthy Natural Language Processing has grown from 8 proceedings papers to 41 over six editions.<n>We synthesize insights from all 144 proceedings papers, classifying them along six trust dimensions grounded in established frameworks.<n>Analysis from the classification study reveals that truthfulness is the fastest-growing dimension (absent in 2021-2022, comprising 37% of papers by 2025-2026)
  arXiv  Detail & Related papers  (2026-08-11T17:30:16Z)
• SciRet: A Compute-Aware Empirical Study of Retrieval and Reranking for Scientific RAG [0.0]
  We introduce SciRet, a compute-aware empirical study of retrieval-augmented generation for scientific question answering over CORD-19.<n>We evaluate a fixed scientific RAG pipeline across three corpus scales: 1,034 chunks (1K papers), 5,160 chunks (5K papers), and 15,480 chunks (15K papers)<n>Across these settings, hybrid retrieval is more robust than either sparse-only or dense-only retrieval in our setting, reaching Recall@10 of 1.000 at 1K and 15K.
  arXiv  Detail & Related papers  (2026-08-04T16:04:07Z)
• Sifei at SemEval-2026 Task 8: Hybrid Retrieval and Query Rewriting for Multi-Turn RAG [2.9721401411878254]
  Multi-turn retrieval-augmented generation (RAG) is challenging due to evolving user intent, conversational noise, and strict context limits.<n>We propose a training-free hybrid retrieval pipeline for SemEval-2026 Task 8 that combines dense and sparse retrieval with controlled query rewriting and cross-encoder reranking.
  arXiv  Detail & Related papers  (2026-06-05T07:28:01Z)
• ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research [142.29356526274387]
  We present ResearchClawBench, a benchmark for evaluating autonomous scientific research.<n>Each task is grounded in a real published paper, provides related literature and raw data, and hides the target paper during evaluation.
  arXiv  Detail & Related papers  (2026-05-28T16:27:40Z)
• Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs [86.49905745865038]
  Soohak is a 439-problem benchmark newly authored from scratch by 64 mathematicians.<n>The dataset will be publicly released in late 2026, with model evaluations available upon request in the interim.
  arXiv  Detail & Related papers  (2026-05-09T17:14:22Z)
• Hybrid Retrieval for COVID-19 Literature: Comparing Rank Fusion and Projection Fusion with Diversity Reranking [0.0]
  We present a hybrid retrieval system for COVID-19 scientific literature, evaluated on the TREC-COVID benchmark.<n>The system implements six retrieval configurations spanning sparse (SPLADE), dense (BGE), rank-level fusion (RRF) and a projection-based vector fusion (B5) approach.<n>RRF fusion achieves the best relevance (nDCG@10 = 0.828), outperforming dense-only by 6.1% and sparse-only by 14.9%.
  arXiv  Detail & Related papers  (2026-04-15T11:05:46Z)
• SAGE: Benchmarking and Improving Retrieval for Deep Research Agents [60.53966065867568]
  We introduce SAGE, a benchmark for scientific literature retrieval comprising 1,200 queries across four scientific domains, with a 200,000 paper retrieval corpus.<n>We evaluate six deep research agents and find that all systems struggle with reasoning-intensive retrieval.<n> BM25 significantly outperforms LLM-based retrievers by approximately 30%, as existing agents generate keyword-oriented sub-queries.
  arXiv  Detail & Related papers  (2026-02-05T18:25:24Z)
• Evaluating Hybrid Retrieval Augmented Generation using Dynamic Test Sets: LiveRAG Challenge [8.680958290253914]
  We present our submission to the LiveRAG Challenge 2025, which evaluates retrieval-augmented generation (RAG) systems on dynamic test sets.<n>Our final hybrid approach combines sparse (BM25) and dense (E5) retrieval methods.<n>We demonstrate that neural re-ranking with RankLLaMA improves MAP from 0.523 to 0.797 but introduces prohibitive computational costs.
  arXiv  Detail & Related papers  (2025-06-27T21:20:43Z)
• Deep Retrieval at CheckThat! 2025: Identifying Scientific Papers from Implicit Social Media Mentions via Hybrid Retrieval and Re-Ranking [4.275139302875217]
  We present the methodology and results of the Deep Retrieval team for subtask 4b of the CLEF CheckThat! 2025 competition.<n>We propose a hybrid retrieval pipeline that combines lexical precision, semantic generalization, and deep contextual re-ranking.<n>Our approach achieves a mean reciprocal rank at 5 (MRR@5) of 76.46% on the development set and 66.43% on the hidden test set.
  arXiv  Detail & Related papers  (2025-05-29T08:55:39Z)
• ReasonIR: Training Retrievers for Reasoning Tasks [139.54343970560103]
  ReasonIR-8B is the first retriever specifically trained for general reasoning tasks.<n>It achieves a new state-of-the-art of 29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a widely-used information retrieval benchmark.
  arXiv  Detail & Related papers  (2025-04-29T09:49:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.