Efficient Multilingual Neural Machine Translation via Corpus-Driven Vocabulary Pruning: An English-Arabic Case Study

• URL: http://arxiv.org/abs/2608.03480v1
• Date: Tue, 04 Aug 2026 11:17:54 GMT
• Title: Efficient Multilingual Neural Machine Translation via Corpus-Driven Vocabulary Pruning: An English-Arabic Case Study
• Abstract summary: We propose a general optimization framework that combines a vocabulary pruning method with a targeted fine-tuning protocol for MNMT models.<n>We evaluate the proposed framework using three models (M2M100, NLLB-200, mBART-50) on the English-Arabic language pair.<n>Results show that optimized multilingual models can match or exceed the performance of dedicated bilingual baselines.
• Score: 0.19921702137386235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The adoption of large pre-trained multilingual models for neural machine translation (MNMT) faces a major challenge: excessive memory and computational consumption due to overly large vocabularies and embedding layers. Although existing compression methods like pruning, quantization and knowledge distillation reduce parameter redundancy, they mainly preserve the structure of the original vocabulary, thereby leaving a major source of inefficiency unresolved. We propose in this paper a general optimization framework that combines a vocabulary pruning method with a targeted fine-tuning protocol for MNMT models. We evaluate the proposed framework using three models (M2M100, NLLB-200, mBART-50) on the English-Arabic language pair. Our approach reduces the vocabulary size from over 128,000 to approximately 10,000 tokens, enabling a 60% memory saving without any loss in performance. Results show that optimized multilingual models can match or exceed the performance of dedicated bilingual baselines. In particular, the pruned and fine-tuned M2M100 model achieves a competitive BLEU score of 42.04 (against 44.59 for the OPUS-MTen- ar bilingual model) while it significantly outperforms it on the COMET metric (0.8730 vs 0.7911) revealing superior semantic adequacy and fluency.

Related papers

• To Each Language Its Tokenizer: Modular Tokenizers for Efficient Multilingual LLMs [15.23810104303108]
  We introduce a modular framework for multilingual model training.<n>First, we propose methods to learn large modular BPE and Unigram tokenizers that enable extraction of subtokenizers tailored to any language subset.<n>Second, we design a pretraining strategy that samples subtokenizers to form batches, restricting predictions to the relevant vocabulary subset and allowing efficient training despite a large vocabulary.
  arXiv  Detail & Related papers  (2026-09-14T13:17:35Z)
• TernaryLM: Memory-Efficient Language Modeling via Native 1-Bit Quantization with Adaptive Layer-wise Scaling [0.39287497907611874]
  We present TernaryLM, a 132M parameter transformer architecture that employs native 1-bit ternary quantization -1, 0, +1 during training.<n>Our results suggest that native 1-bit training is a promising direction for efficient neural language models.
  arXiv  Detail & Related papers  (2026-02-07T05:35:17Z)
• Machine Translation for Ge'ez Language [0.0]
  Machine translation for low-resource languages such as Ge'ez faces challenges such as out-of-vocabulary words, domain mismatches, and lack of labeled training data. We develop a multilingual neural machine translation (MNMT) model based on languages relatedness. We also experiment with using GPT-3.5, a state-of-the-art LLM, for few-shot translation with fuzzy matches.
  arXiv  Detail & Related papers  (2023-11-24T14:55:23Z)
• Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model [92.91310997807936]
  NLLB-200 is a set of multilingual Neural Machine Translation models that cover 202 languages. We propose a pruning method that enables the removal of up to 80% of experts without further finetuning.
  arXiv  Detail & Related papers  (2022-12-19T19:29:40Z)
• SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages [102.50127671423752]
  We introduce SMaLL-100, a distilled version of the M2M-100 (12B) machine translation model covering 100 languages. We train SMaLL-100 with uniform sampling across all language pairs and therefore focus on preserving the performance of low-resource languages. Our model achieves comparable results to M2M-100 (1.2B), while being 3.6x smaller and 4.3x faster at inference.
  arXiv  Detail & Related papers  (2022-10-20T22:32:29Z)
• Distributionally Robust Multilingual Machine Translation [94.51866646879337]
  We propose a new learning objective for Multilingual neural machine translation (MNMT) based on distributionally robust optimization. We show how to practically optimize this objective for large translation corpora using an iterated best response scheme. Our method consistently outperforms strong baseline methods in terms of average and per-language performance under both many-to-one and one-to-many translation settings.
  arXiv  Detail & Related papers  (2021-09-09T03:48:35Z)
• Neural Machine Translation with Monolingual Translation Memory [58.98657907678992]
  We propose a new framework that uses monolingual memory and performs learnable memory retrieval in a cross-lingual manner. Experiments show that the proposed method obtains substantial improvements.
  arXiv  Detail & Related papers  (2021-05-24T13:35:19Z)
• Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information [72.2412707779571]
  mRASP is an approach to pre-train a universal multilingual neural machine translation model. We carry out experiments on 42 translation directions across a diverse setting, including low, medium, rich resource, and as well as transferring to exotic language pairs.
  arXiv  Detail & Related papers  (2020-10-07T03:57:54Z)
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation [81.7786241489002]
  Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics. We propose random online backtranslation to enforce the translation of unseen training language pairs.
  arXiv  Detail & Related papers  (2020-04-24T17:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.