Lost in Transliteration: Bridging the Script Gap in Neural IR

• URL: http://arxiv.org/abs/2505.08411v1
• Date: Tue, 13 May 2025 10:09:51 GMT
• Title: Lost in Transliteration: Bridging the Script Gap in Neural IR
• Authors: Andreas Chari, Iadh Ounis, Sean MacAvaney,
• Abstract summary: This paper shows that current search systems, including those that use multilingual dense embeddings, do not generalise to transliterated queries.<n>We explore whether adapting the popular translate-train" paradigm to transliterations can enhance the robustness of multilingual Information Retrieval methods.
• Score: 23.572881425446074
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most human languages use scripts other than the Latin alphabet. Search users in these languages often formulate their information needs in a transliterated -- usually Latinized -- form for ease of typing. For example, Greek speakers might use Greeklish, and Arabic speakers might use Arabizi. This paper shows that current search systems, including those that use multilingual dense embeddings such as BGE-M3, do not generalise to this setting, and their performance rapidly deteriorates when exposed to transliterated queries. This creates a ``script gap" between the performance of the same queries when written in their native or transliterated form. We explore whether adapting the popular ``translate-train" paradigm to transliterations can enhance the robustness of multilingual Information Retrieval (IR) methods and bridge the gap between native and transliterated scripts. By exploring various combinations of non-Latin and Latinized query text for training, we investigate whether we can enhance the capacity of existing neural retrieval techniques and enable them to apply to this important setting. We show that by further fine-tuning IR models on an even mixture of native and Latinized text, they can perform this cross-script matching at nearly the same performance as when the query was formulated in the native script. Out-of-domain evaluation and further qualitative analysis show that transliterations can also cause queries to lose some of their nuances, motivating further research in this direction.

Related papers

• SoftMatcha: A Soft and Fast Pattern Matcher for Billion-Scale Corpus Searches [5.80278230280824]
  We propose a novel algorithm that achieves semantic yet efficient pattern matching by relaxing a surface-level matching with word embeddings.<n>Our experiments demonstrate that the proposed method can execute searches on billion-scale corpora in less than a second.
  arXiv  Detail & Related papers  (2025-03-05T17:53:11Z)
• mFollowIR: a Multilingual Benchmark for Instruction Following in Retrieval [61.17793165194077]
  We introduce mFollowIR, a benchmark for measuring instruction-following ability in retrieval models.<n>We present results for both multilingual (XX-XX) and cross-lingual (En-XX) performance.<n>We see strong cross-lingual performance with English-based retrievers that trained using instructions, but find a notable drop in performance in the multilingual setting.
  arXiv  Detail & Related papers  (2025-01-31T16:24:46Z)
• Exploring the Role of Transliteration in In-Context Learning for Low-resource Languages Written in Non-Latin Scripts [50.40191599304911]
  We investigate whether transliteration is also effective in improving LLMs' performance for low-resource languages written in non-Latin scripts. We propose three prompt templates, where the target-language text is represented in (1) its original script, (2) Latin script, or (3) both. Our findings show that the effectiveness of transliteration varies by task type and model size.
  arXiv  Detail & Related papers  (2024-07-02T14:51:20Z)
• Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment [50.27950279695363]
  The transfer performance is often hindered when a low-resource target language is written in a different script than the high-resource source language. Inspired by recent work that uses transliteration to address this problem, our paper proposes a transliteration-based post-pretraining alignment (PPA) method.
  arXiv  Detail & Related papers  (2024-06-28T08:59:24Z)
• Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languages [8.858671209228536]
  We investigate lexical sharing in multilingual machine translation from Hindi, Gujarati, Nepali into English. We find that transliteration does not give pronounced improvements. Our analysis suggests that our multilingual MT models trained on original scripts seem to already be robust to cross-script differences.
  arXiv  Detail & Related papers  (2023-05-04T23:35:15Z)
• Romanization-based Large-scale Adaptation of Multilingual Language Models [124.57923286144515]
  Large multilingual pretrained language models (mPLMs) have become the de facto state of the art for cross-lingual transfer in NLP. We study and compare a plethora of data- and parameter-efficient strategies for adapting the mPLMs to romanized and non-romanized corpora of 14 diverse low-resource languages. Our results reveal that UROMAN-based transliteration can offer strong performance for many languages, with particular gains achieved in the most challenging setups.
  arXiv  Detail & Related papers  (2023-04-18T09:58:34Z)
• Does Transliteration Help Multilingual Language Modeling? [0.0]
  We empirically measure the effect of transliteration on Multilingual Language Models. We focus on the Indic languages, which have the highest script diversity in the world. We find that transliteration benefits the low-resource languages without negatively affecting the comparatively high-resource languages.
  arXiv  Detail & Related papers  (2022-01-29T05:48:42Z)
• Mixed Attention Transformer for LeveragingWord-Level Knowledge to Neural Cross-Lingual Information Retrieval [15.902630454568811]
  We propose a novel Mixed Attention Transformer (MAT) that incorporates external word level knowledge, such as a dictionary or translation table. By encoding the translation knowledge into an attention matrix, the model with MAT is able to focus on the mutually translated words in the input sequence.
  arXiv  Detail & Related papers  (2021-09-07T00:33:14Z)
• A Simple and Efficient Probabilistic Language model for Code-Mixed Text [0.0]
  We present a simple probabilistic approach for building efficient word embedding for code-mixed text. We examine its efficacy for the classification task using bidirectional LSTMs and SVMs.
  arXiv  Detail & Related papers  (2021-06-29T05:37:57Z)
• VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation [77.82373082024934]
  We plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. The proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark.
  arXiv  Detail & Related papers  (2020-10-30T03:41:38Z)
• Intrinsic Probing through Dimension Selection [69.52439198455438]
  Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of tasks. Such high performance should not be possible unless some form of linguistic structure inheres in these representations, and a wealth of research has sprung up on probing for it. In this paper, we draw a distinction between intrinsic probing, which examines how linguistic information is structured within a representation, and the extrinsic probing popular in prior work, which only argues for the presence of such information by showing that it can be successfully extracted.
  arXiv  Detail & Related papers  (2020-10-06T15:21:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.