Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methods

• URL: http://arxiv.org/abs/2509.06195v1
• Date: Sun, 07 Sep 2025 20:10:49 GMT
• Title: Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methods
• Abstract summary: Language fairness in multilingual information retrieval (MLIR) systems is crucial for ensuring equitable access to information across diverse languages.<n>This paper sheds light on the issue, based on the assumption that queries in different languages, but with identical semantics, should yield equivalent ranking lists when retrieving on the same multilingual documents.<n>We evaluate the degree of fairness using both traditional retrieval methods, and a DPR neural ranker based on mBERT and XLM-R.
• Score: 34.309727925633354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language fairness in multilingual information retrieval (MLIR) systems is crucial for ensuring equitable access to information across diverse languages. This paper sheds light on the issue, based on the assumption that queries in different languages, but with identical semantics, should yield equivalent ranking lists when retrieving on the same multilingual documents. We evaluate the degree of fairness using both traditional retrieval methods, and a DPR neural ranker based on mBERT and XLM-R. Additionally, we introduce `LaKDA', a novel loss designed to mitigate language biases in neural MLIR approaches. Our analysis exposes intrinsic language biases in current MLIR technologies, with notable disparities across the retrieval methods, and the effectiveness of LaKDA in enhancing language fairness.

Related papers

• Translation or Recitation? Calibrating Evaluation Scores for Machine Translation of Extremely Low-Resource Languages [39.985923582735936]
  We introduce the FRED Difficulty Metrics, which include the Fertility Ratio (F), Retrieval Proxy (R), Pre-training Exposure (E) and Corpus Diversity (D)<n>These metrics reveal that a significant portion of result variability is explained by train-test overlap and pre-training exposure rather than model capability.
  arXiv  Detail & Related papers  (2026-03-26T09:20:17Z)
• What Drives Cross-lingual Ranking? Retrieval Approaches with Multilingual Language Models [0.19116784879310025]
  Cross-lingual information retrieval is challenging due to disparities in resources, scripts, and weak cross-lingual semantic alignment in embedding models.<n>Existing pipelines often rely on translation and monolingual retrievals, which add computational overhead and noise, performance.<n>This work systematically evaluates four intervention types, namely document translation, multilingual dense retrieval with pretrained encoders, contrastive learning at word, phrase, and query-document levels, and cross-encoder re-ranking, across three benchmark datasets.
  arXiv  Detail & Related papers  (2025-11-24T17:17:40Z)
• Bridging Language Gaps: Advances in Cross-Lingual Information Retrieval with Multilingual LLMs [0.19116784879310025]
  Cross-lingual information retrieval (CLIR) addresses the challenge of retrieving relevant documents written in languages different from that of the original query.<n>Recent advances have shifted from translation-based methods toward embedding-based approaches.<n>This survey provides a comprehensive overview of developments from early translation-based methods to state-of-the-art embedding-driven and generative techniques.
  arXiv  Detail & Related papers  (2025-10-01T13:50:05Z)
• Beyond WER: Probing Whisper's Sub-token Decoder Across Diverse Language Resource Levels [6.627057618324123]
  This paper introduces a fine-grained analysis of Whisper's multilingual decoder.<n>Our method traces the beam search path, capturing sub-token guesses and their associated probabilities.<n>Lower resource languages fare worse on these metrics, but also exhibit distinct clustering patterns in sub-token usage.
  arXiv  Detail & Related papers  (2025-09-29T21:20:05Z)
• Investigating Language and Retrieval Bias in Multilingual Previously Fact-Checked Claim Detection [4.6738956348193]
  Large Language Models (LLMs) offer powerful capabilities for cross-lingual fact-checking.<n>LLMs often exhibit language bias, performing disproportionately better on high-resource languages such as English.<n>We present and inspect a novel concept - retrieval bias, when information retrieval systems tend to favor certain information over others.
  arXiv  Detail & Related papers  (2025-09-29T17:50:32Z)
• How and Where to Translate? The Impact of Translation Strategies in Cross-lingual LLM Prompting [15.388822834013599]
  In multilingual retrieval-augmented generation (RAG)-based systems, knowledge bases (KB) are often shared from high-resource languages (such as English) to low-resource ones.<n>Two common practices are pre-translation to create a mono-lingual prompt and cross-lingual prompting for direct inference.<n>We show that an optimized prompting strategy can significantly improve knowledge sharing across languages, therefore improve the performance on the downstream classification task.
  arXiv  Detail & Related papers  (2025-07-21T19:37:15Z)
• Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Task [73.35882908048423]
  Retrieval-augmented generation (RAG) has become a cornerstone of contemporary NLP.<n>This paper investigates the effectiveness of RAG across multiple languages by proposing novel approaches for multilingual open-domain question-answering.
  arXiv  Detail & Related papers  (2025-04-04T17:35:43Z)
• Multilingual Retrieval Augmented Generation for Culturally-Sensitive Tasks: A Benchmark for Cross-lingual Robustness [30.00463676754559]
  We introduce BordIRLines, a dataset of territorial disputes paired with retrieved Wikipedia documents, across 49 languages.<n>We evaluate the cross-lingual robustness of this RAG setting by formalizing several modes for multilingual retrieval.<n>Our experiments show that incorporating perspectives from diverse languages can in fact improve robustness.
  arXiv  Detail & Related papers  (2024-10-02T01:59:07Z)
• Language Fairness in Multilingual Information Retrieval [9.358582006140903]
  This work proposes a language fairness metric to evaluate whether documents across different languages are fairly ranked. In contrast to most prior work in group fairness, we do not consider any language to be an unprotected group. Our proposed measure, PEER, is the first fairness metric specifically designed to capture the language fairness of MLIR systems.
  arXiv  Detail & Related papers  (2024-05-02T03:30:15Z)
• Cross-lingual Text Classification Transfer: The Case of Ukrainian [11.508759658889382]
  Ukrainian stands as a language that can benefit from the continued refinement of cross-lingual methodologies.<n>Due to our knowledge, there is a tremendous lack of Ukrainian corpora for typical text classification tasks.<n>In this work, we leverage the state-of-the-art advances in NLP, exploring cross-lingual knowledge transfer methods.
  arXiv  Detail & Related papers  (2024-04-02T15:37:09Z)
• A Multi-level Supervised Contrastive Learning Framework for Low-Resource Natural Language Inference [54.678516076366506]
  Natural Language Inference (NLI) is a growingly essential task in natural language understanding. Here we propose a multi-level supervised contrastive learning framework named MultiSCL for low-resource natural language inference.
  arXiv  Detail & Related papers  (2022-05-31T05:54:18Z)
• Exposing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders [85.80950708769923]
  We probe multilingual language models for the amount of cross-lingual lexical knowledge stored in their parameters, and compare them against the original multilingual LMs. We also devise a novel method to expose this knowledge by additionally fine-tuning multilingual models. We report substantial gains on standard benchmarks.
  arXiv  Detail & Related papers  (2022-04-30T13:23:16Z)
• On Cross-Lingual Retrieval with Multilingual Text Encoders [51.60862829942932]
  We study the suitability of state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks. We benchmark their performance in unsupervised ad-hoc sentence- and document-level CLIR experiments. We evaluate multilingual encoders fine-tuned in a supervised fashion (i.e., we learn to rank) on English relevance data in a series of zero-shot language and domain transfer CLIR experiments.
  arXiv  Detail & Related papers  (2021-12-21T08:10:27Z)
• A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representations [7.571549274473274]
  Language agnostic and semantic-language information isolation is an emerging research direction for multilingual representations models. "Language Information Removal (LIR)" factors out language identity information from semantic related components in multilingual representations pre-trained on multi-monolingual data. LIR reveals that for weak-alignment multilingual systems, the principal components of semantic spaces primarily encodes language identity information.
  arXiv  Detail & Related papers  (2021-09-10T08:15:37Z)
• On the Language Coverage Bias for Neural Machine Translation [81.81456880770762]
  Language coverage bias is important for neural machine translation (NMT) because the target-original training data is not well exploited in current practice. By carefully designing experiments, we provide comprehensive analyses of the language coverage bias in the training data. We propose two simple and effective approaches to alleviate the language coverage bias problem.
  arXiv  Detail & Related papers  (2021-06-07T01:55:34Z)
• Decoding and Diversity in Machine Translation [90.33636694717954]
  We characterize differences between cost diversity paid for the BLEU scores enjoyed by NMT. Our study implicates search as a salient source of known bias when translating gender pronouns.
  arXiv  Detail & Related papers  (2020-11-26T21:09:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.