Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya

• URL: http://arxiv.org/abs/2607.15209v1
• Date: Thu, 16 Jul 2026 17:08:28 GMT
• Title: Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya
• Abstract summary: We introduce a vocabulary-extended variant of XLM-R targeting the two highest-resource Ge'ez-script languages, Amharic and Tigrinya.<n>VexMLM achieves 87.0 EM /90.0 F1, versus 66.0 EM/78.0 F1 for XLM-R and 74.0 EM/ 78.0 F1 for Glot500.<n>On SA, VEXMLM reaches 80.0% accuracy versus 77.0% (XLM-R) and 46.0% (Glot500)
• Score: 6.396476220617138
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multilingual pre-trained language models (PLMs) exhibit degraded performance on low-resource, non-Latin-script languages, driven by high out-of-vocabulary (OOV) rates and excessive subword fragmentation that result from Latin-script-centric tokenizer training. We introduce VEXMLM, a vocabulary-extended variant of XLM-R targeting the two highest-resource Ge'ez-script languages, Amharic and Tigrinya, and further evaluated on 17 additional low-resource African languages (19 total). We train a language-specific SentencePiece tokenizer on curated Amharic and Tigrinya monolingual corpora, extend XLM-R's vocabulary with 30,000 Ge'ez-script subwords derived from this tokenizer, and initialize their embeddings by averaging the embeddings of their constituent subwords under XLM-R's original tokenizer. VEXMLM is trained in two stages: (1) continued masked language modeling over the extended vocabulary on the curated corpora, and (2) supervised fine-tuning on question answering (QA), named entity recognition (NER), and sentiment analysis (SA). On Amharic/Tigrinya QA, VEXMLM achieves 87.0 EM /90.0 F1, versus 66.0 EM/78.0 F1 for XLM-R and 74.0 EM/ 78.0 F1 for Glot500. On SA, VEXMLM reaches 80.0\% accuracy versus 77.0\% (XLM-R) and 46.0\% (Glot500). On NER, VEXMLM raises OOV-token entity accuracy from 81.4\% to 94.3\%, averaged over 11 of the 19 evaluated languages for which OOV analysis was possible. Our contributions are: (i) a vocabulary-extension and embedding-initialization procedure tailored to Ge'ez script; (ii) a two-stage training strategy under which vocabulary and continued-pretraining gains on Amharic/Tigrinya transfer to 17 typologically related, unaugmented African languages; and (iii) an evaluation spanning both intrinsic tokenization metrics (vocabulary coverage, fertility, OOV rate) and extrinsic task performance across all 19 languages.

Related papers

• Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech [61.759910921200834]
  Cross-lingual sentence encoders typically cover only a few hundred languages.<n>We introduce OmniSONAR, a new family of omnilingual, cross-lingual and cross-language sentence embedding models.
  arXiv  Detail & Related papers  (2026-03-17T14:47:35Z)
• TriLex: A Framework for Multilingual Sentiment Analysis in Low-Resource South African Languages [0.0]
  Low-resource African languages remain underrepresented in sentiment analysis.<n>This study proposes TriLex, a three-stage retrieval augmented framework that unifies corpus-based extraction, cross lingual mapping, and RAG driven lexical refinement.<n>Using the enriched lexicon, the performance of two prominent African pretrained language models is evaluated.
  arXiv  Detail & Related papers  (2025-12-02T14:16:31Z)
• IndicSuperTokenizer: An Optimized Tokenizer for Indic Multilingual LLMs [5.068673710249497]
  IndicSuperTokenizer is a tokenizer for Indic multilingual LLMs.<n>It combines subword and multi-word tokenization, along with language-specific tokens pre-tokenization.<n>It improves the average fertility score by 39.5% over LLaMA4 and by 18% over Sutra.
  arXiv  Detail & Related papers  (2025-11-05T06:57:42Z)
• One ruler to measure them all: Benchmarking multilingual long-context language models [31.669282956425658]
  We present ONERULER, a multilingual benchmark designed to evaluate long-context language models across 26 languages.<n>English is not the top-performing language on long-context tasks (ranked 6th out of 26), with Polish emerging as the top language.<n>In cross-lingual scenarios where instructions and context appear in different languages, performance can fluctuate by up to 20% depending on the instruction language.
  arXiv  Detail & Related papers  (2025-03-03T19:12:48Z)
• Franken-Adapter: Cross-Lingual Adaptation of LLMs by Embedding Surgery [31.516243610548635]
  We present $textitFranken-Adapter$, a modular language adaptation approach for decoder-only Large Language Models.<n>Our method begins by creating customized vocabularies for target languages and performing language adaptation through embedding tuning on multilingual data.<n>Experiments on $ttGemma2$ models with up to 27B parameters demonstrate improvements of up to 20% across 96 languages, spanning both discriminative and generative tasks.
  arXiv  Detail & Related papers  (2025-02-12T00:38:11Z)
• Enhancing Multilingual ASR for Unseen Languages via Language Embedding Modeling [50.62091603179394]
  Whisper, one of the most advanced ASR models, handles 99 languages effectively.<n>However, Whisper struggles with unseen languages, those not included in its pre-training.<n>We propose methods that exploit these relationships to enhance ASR performance on unseen languages.
  arXiv  Detail & Related papers  (2024-12-21T04:05:43Z)
• Prompting with Phonemes: Enhancing LLMs' Multilinguality for Non-Latin Script Languages [37.61699757912346]
  We propose leveraging phonemic transcriptions as complementary signals to induce script-invariant representations.<n>We show that phonemic and orthographic scripts retrieve distinct examples for in-context learning (ICL)<n>This motivates our proposed Mixed-ICL retrieval strategy, where further aggregation from both leads to significant performance improvements.
  arXiv  Detail & Related papers  (2024-11-04T18:59:51Z)
• Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations [59.056367787688146]
  This paper pioneers exploring and training powerful Multilingual Math Reasoning (xMR) LLMs. We construct the first multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages. By utilizing translation, we construct the first multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages.
  arXiv  Detail & Related papers  (2023-10-31T08:09:20Z)
• Extrapolating Large Language Models to Non-English by Aligning Languages [109.09051737966178]
  Existing large language models show disparate capability across different languages. In this paper, we empower pre-trained LLMs on non-English languages by building semantic alignment across languages.
  arXiv  Detail & Related papers  (2023-08-09T13:32:06Z)
• XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models [100.29953199404905]
  We introduce a new approach for scaling to very large multilingual vocabularies by de-emphasizing token sharing between languages with little lexical overlap. We train XLM-V, a multilingual language model with a one million token vocabulary. XLM-V is particularly effective on low-resource language tasks and outperforms XLM-R by 11.2% and 5.8% absolute on MasakhaNER and Americas NLI, respectively.
  arXiv  Detail & Related papers  (2023-01-25T09:15:17Z)
• AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages [75.08199398141744]
  We present AmericasNLI, an extension of XNLI (Conneau et al.), to 10 indigenous languages of the Americas. We conduct experiments with XLM-R, testing multiple zero-shot and translation-based approaches. We find that XLM-R's zero-shot performance is poor for all 10 languages, with an average performance of 38.62%.
  arXiv  Detail & Related papers  (2021-04-18T05:32:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.