To Case or Not to Case: An Empirical Study in Learned Sparse Retrieval

• URL: http://arxiv.org/abs/2601.17500v1
• Date: Sat, 24 Jan 2026 15:58:10 GMT
• Title: To Case or Not to Case: An Empirical Study in Learned Sparse Retrieval
• Authors: Emmanouil Georgios Lionis, Jia-Huei Ju, Angelos Nalmpantis, Casper Thuis, Sean MacAvaney, Andrew Yates,
• Abstract summary: Learned Sparse Retrieval (LSR) methods construct sparse lexical representations of queries and documents.<n>Existing LSR approaches have relied almost exclusively on uncased backbone models.<n>Cased models almost entirely suppress cased vocabulary items and behave effectively as uncased models.
• Score: 25.242514696943616
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learned Sparse Retrieval (LSR) methods construct sparse lexical representations of queries and documents that can be efficiently searched using inverted indexes. Existing LSR approaches have relied almost exclusively on uncased backbone models, whose vocabularies exclude case-sensitive distinctions, thereby reducing vocabulary mismatch. However, the most recent state-of-the-art language models are only available in cased versions. Despite this shift, the impact of backbone model casing on LSR has not been studied, potentially posing a risk to the viability of the method going forward. To fill this gap, we systematically evaluate paired cased and uncased versions of the same backbone models across multiple datasets to assess their suitability for LSR. Our findings show that LSR models with cased backbone models by default perform substantially worse than their uncased counterparts; however, this gap can be eliminated by pre-processing the text to lowercase. Moreover, our token-level analysis reveals that, under lowercasing, cased models almost entirely suppress cased vocabulary items and behave effectively as uncased models, explaining their restored performance. This result broadens the applicability of recent cased models to the LSR setting and facilitates the integration of stronger backbone architectures into sparse retrieval. The complete code and implementation for this project are available at: https://github.com/lionisakis/Uncased-vs-cased-models-in-LSR

Related papers

• A Study on Building Efficient Zero-Shot Relation Extraction Models [5.82696370413126]
  We study the robustness of existing zero-shot relation extraction models when adapting them to a realistic extraction scenario.<n>We adapt several state-of-the-art tools, and compare them in this challenging setting.
  arXiv  Detail & Related papers  (2026-03-01T20:44:31Z)
• Bi'an: A Bilingual Benchmark and Model for Hallucination Detection in Retrieval-Augmented Generation [6.549143816134529]
  We introduce bftextBi'an, a novel framework featuring a bilingual benchmark dataset and lightweight judge models.<n>The dataset supports rigorous evaluation across multiple RAG scenarios, while the judge models are fine-tuned from compact open-source LLMs.
  arXiv  Detail & Related papers  (2025-02-26T15:12:59Z)
• Predicting the Performance of Black-box LLMs through Self-Queries [60.87193950962585]
  Large language models (LLMs) are increasingly relied on in AI systems, predicting when they make mistakes is crucial.<n>In this paper, we extract features of LLMs in a black-box manner by using follow-up prompts and taking the probabilities of different responses as representations.<n>We demonstrate that training a linear model on these low-dimensional representations produces reliable predictors of model performance at the instance level.
  arXiv  Detail & Related papers  (2025-01-02T22:26:54Z)
• Unraveling and Mitigating Retriever Inconsistencies in Retrieval-Augmented Large Language Models [25.346800371132645]
  Retrieval-augmented Large Language Models (RALMs) do not consistently outperform the original retrieval-free Language Models (LMs)<n>Our experiments reveal that this example-level performance inconsistency exists not only between retrieval-augmented and retrieval-free LM but also among different retrievers.<n>We introduce Ensemble of Retrievers (EoR), a trainable framework that can adaptively retrieve from different knowledge sources and effectively decrease unpredictable reader errors.
  arXiv  Detail & Related papers  (2024-05-31T08:22:49Z)
• LLM-Augmented Retrieval: Enhancing Retrieval Models Through Language Models and Doc-Level Embedding [2.0257616108612373]
  This paper introduces a model-agnostic doc-level embedding framework through large language model augmentation. We have been able to significantly improve the effectiveness of widely-used retriever models.
  arXiv  Detail & Related papers  (2024-04-08T19:29:07Z)
• Making Retrieval-Augmented Language Models Robust to Irrelevant Context [55.564789967211844]
  An important desideratum of RALMs, is that retrieved information helps model performance when it is relevant. Recent work has shown that retrieval augmentation can sometimes have a negative effect on performance.
  arXiv  Detail & Related papers  (2023-10-02T18:52:35Z)
• HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models [81.56455625624041]
  We introduce the first open-source benchmark to utilize external large language models (LLMs) for ASR error correction. The proposed benchmark contains a novel dataset, HyPoradise (HP), encompassing more than 334,000 pairs of N-best hypotheses. LLMs with reasonable prompt and its generative capability can even correct those tokens that are missing in N-best list.
  arXiv  Detail & Related papers  (2023-09-27T14:44:10Z)
• SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval [92.27387459751309]
  We provide SPRINT, a unified Python toolkit for evaluating neural sparse retrieval. We establish strong and reproducible zero-shot sparse retrieval baselines across the well-acknowledged benchmark, BEIR. We show that SPLADEv2 produces sparse representations with a majority of tokens outside of the original query and document.
  arXiv  Detail & Related papers  (2023-07-19T22:48:02Z)
• Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models [6.425088990363101]
  We examine the relationship between fluency and attribution in Large Language Models prompted with retrieved evidence. We show that larger models tend to do much better in both fluency and attribution. We propose a recipe that could allow smaller models to both close the gap with larger models and preserve the benefits of top-k retrieval.
  arXiv  Detail & Related papers  (2023-02-11T02:43:34Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• Self-Damaging Contrastive Learning [92.34124578823977]
  Unlabeled data in reality is commonly imbalanced and shows a long-tail distribution. This paper proposes a principled framework called Self-Damaging Contrastive Learning to automatically balance the representation learning without knowing the classes. Our experiments show that SDCLR significantly improves not only overall accuracies but also balancedness.
  arXiv  Detail & Related papers  (2021-06-06T00:04:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.