A Case Against Implicit Standards: Homophone Normalization in Machine Translation for Languages that use the Ge'ez Script

• URL: http://arxiv.org/abs/2507.15142v1
• Date: Sun, 20 Jul 2025 22:35:08 GMT
• Title: A Case Against Implicit Standards: Homophone Normalization in Machine Translation for Languages that use the Ge'ez Script
• Authors: Hellina Hailu Nigatu, Atnafu Lambebo Tonja, Henok Biadglign Ademtew, Hizkel Mitiku Alemayehu, Negasi Haile Abadi, Tadesse Destaw Belay, Seid Muhie Yimam,
• Abstract summary: Homophone normalization is a pre-processing step applied in Amharic Natural Language Processing literature.<n>We propose a post-inference intervention in which normalization is applied to model predictions instead of training data.<n>Our work contributes to the broader discussion on technology-facilitated language change and calls for more language-aware interventions.
• Score: 3.5149312379702127
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Homophone normalization, where characters that have the same sound in a writing script are mapped to one character, is a pre-processing step applied in Amharic Natural Language Processing (NLP) literature. While this may improve performance reported by automatic metrics, it also results in models that are not able to understand different forms of writing in a single language. Further, there might be impacts in transfer learning, where models trained on normalized data do not generalize well to other languages. In this paper, we experiment with monolingual training and cross-lingual transfer to understand the impacts of normalization on languages that use the Ge'ez script. We then propose a post-inference intervention in which normalization is applied to model predictions instead of training data. With our simple scheme of post-inference normalization, we show that we can achieve an increase in BLEU score of up to 1.03 while preserving language features in training. Our work contributes to the broader discussion on technology-facilitated language change and calls for more language-aware interventions.

Related papers

• What do self-supervised speech models know about Dutch? Analyzing advantages of language-specific pre-training [2.8038082486377114]
  Existing work has shown that a range of linguistic features can be successfully decoded from end-to-end models trained only on speech recordings.<n>Here we test the encoding of Dutch phonetic and lexical information in internal representations of self-supervised Wav2Vec2 models.
  arXiv  Detail & Related papers  (2025-06-01T12:25:13Z)
• Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment [50.27950279695363]
  The transfer performance is often hindered when a low-resource target language is written in a different script than the high-resource source language. Inspired by recent work that uses transliteration to address this problem, our paper proposes a transliteration-based post-pretraining alignment (PPA) method.
  arXiv  Detail & Related papers  (2024-06-28T08:59:24Z)
• We're Calling an Intervention: Exploring Fundamental Hurdles in Adapting Language Models to Nonstandard Text [8.956635443376527]
  We present a suite of experiments that allow us to understand the underlying challenges of language model adaptation to nonstandard text.<n>We do so by designing interventions that approximate core features of user-generated text and their interactions with existing biases of language models.<n>Applying our interventions during language model adaptation to nonstandard text variations, we gain important insights into when such adaptation is successful.
  arXiv  Detail & Related papers  (2024-04-10T18:56:53Z)
• Machine Translation to Control Formality Features in the Target Language [0.9208007322096532]
  This research explores how machine learning methods are used to translate from English to languages with formality. It was done by training a bilingual model in a formality-controlled setting and comparing its performance with a pre-trained multilingual model. We evaluate the official formality accuracy(ACC) by comparing the predicted masked tokens with the ground truth.
  arXiv  Detail & Related papers  (2023-11-22T15:42:51Z)
• Soft Language Clustering for Multilingual Model Pre-training [57.18058739931463]
  We propose XLM-P, which contextually retrieves prompts as flexible guidance for encoding instances conditionally. Our XLM-P enables (1) lightweight modeling of language-invariant and language-specific knowledge across languages, and (2) easy integration with other multilingual pre-training methods.
  arXiv  Detail & Related papers  (2023-06-13T08:08:08Z)
• Script Normalization for Unconventional Writing of Under-Resourced Languages in Bilingual Communities [36.578851892373365]
  Social media has provided linguistically under-represented communities with an extraordinary opportunity to create content in their native languages. This paper addresses the problem of script normalization for several such languages that are mainly written in a Perso-Arabic script. Using synthetic data with various levels of noise and a transformer-based model, we demonstrate that the problem can be effectively remediated.
  arXiv  Detail & Related papers  (2023-05-25T18:18:42Z)
• Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages [18.210880703295253]
  We finetune pretrained language models (PLMs) on seven languages from three different families. We analyze their zero-shot performance on closely related, non-standardized varieties. Overall, we find that the similarity between the percentage of words that get split into subwords in the source and target data is the strongest predictor for model performance on target data.
  arXiv  Detail & Related papers  (2023-04-20T08:32:34Z)
• Lifting the Curse of Multilinguality by Pre-training Modular Transformers [72.46919537293068]
  multilingual pre-trained models suffer from the curse of multilinguality, which causes per-language performance to drop as they cover more languages. We introduce language-specific modules, which allows us to grow the total capacity of the model, while keeping the total number of trainable parameters per language constant. Our approach enables adding languages post-hoc with no measurable drop in performance, no longer limiting the model usage to the set of pre-trained languages.
  arXiv  Detail & Related papers  (2022-05-12T17:59:56Z)
• Towards Zero-shot Language Modeling [90.80124496312274]
  We construct a neural model that is inductively biased towards learning human languages. We infer this distribution from a sample of typologically diverse training languages. We harness additional language-specific side information as distant supervision for held-out languages.
  arXiv  Detail & Related papers  (2021-08-06T23:49:18Z)
• Cross-lingual Spoken Language Understanding with Regularized Representation Alignment [71.53159402053392]
  We propose a regularization approach to align word-level and sentence-level representations across languages without any external resource. Experiments on the cross-lingual spoken language understanding task show that our model outperforms current state-of-the-art methods in both few-shot and zero-shot scenarios.
  arXiv  Detail & Related papers  (2020-09-30T08:56:53Z)
• On the Importance of Word Order Information in Cross-lingual Sequence Labeling [80.65425412067464]
  Cross-lingual models that fit into the word order of the source language might fail to handle target languages. We investigate whether making models insensitive to the word order of the source language can improve the adaptation performance in target languages.
  arXiv  Detail & Related papers  (2020-01-30T03:35:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.