Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models

• URL: http://arxiv.org/abs/2602.11961v1
• Date: Thu, 12 Feb 2026 13:56:02 GMT
• Title: Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models
• Authors: Yuzhe Shang, Pengzhi Gao, Wei Liu, Jian Luan, Jinsong Su,
• Abstract summary: We present a study of open large language models (LLMs) for multilingual machine translation (MT) across a range of languages.<n>We develop MiLMMT-46, which achieves top-tier multilingual translation performance across 46 languages.
• Score: 38.626788948945844
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Open large language models (LLMs) have demonstrated improving multilingual capabilities in recent years. In this paper, we present a study of open LLMs for multilingual machine translation (MT) across a range of languages, and investigate the effects of model scaling and data scaling when adapting open LLMs to multilingual MT through continual pretraining and instruction finetuning. Based on the Gemma3 model family, we develop MiLMMT-46, which achieves top-tier multilingual translation performance across 46 languages. Extensive experiments show that MiLMMT-46 consistently outperforms recent state-of-the-art (SOTA) models, including Seed-X, HY-MT-1.5, and TranslateGemma, and achieves competitive performance with strong proprietary systems such as Google Translate and Gemini 3 Pro.

Related papers

• Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters [53.59868121093848]
  We introduce Seed-X, a family of open-source language models (LLMs) with 7B parameter size.<n>The base model is pre-trained on a diverse, high-quality dataset encompassing both monolingual and bilingual content across 28 languages.<n>The instruct model is then finetuned to translate by Chain-of-Thought (CoT) reasoning and further enhanced through reinforcement learning (RL) to achieve better generalization across diverse language pairs.
  arXiv  Detail & Related papers  (2025-07-18T03:19:43Z)
• M-Prometheus: A Suite of Open Multilingual LLM Judges [65.63858815169924]
  We introduce M-Prometheus, a suite of open-weight LLM judges that can provide both direct assessment and pairwise comparison feedback on multilingual outputs.<n>M-Prometheus models outperform state-of-the-art open LLM judges on multilingual reward benchmarks spanning more than 20 languages, as well as on literary machine translation (MT) evaluation covering 4 language pairs.
  arXiv  Detail & Related papers  (2025-04-07T11:37:26Z)
• Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study [13.409987421121405]
  GemmaX2-28 is a 9B model achieving top-tier multilingual translation performance across 28 languages.<n>GemmaX2-28 consistently outperforms the state-of-the-art (SOTA) models such as TowerInstruct and XALMA.
  arXiv  Detail & Related papers  (2025-02-04T16:57:03Z)
• Domain-Specific Translation with Open-Source Large Language Models: Resource-Oriented Analysis [0.0]
  We compare the domain-specific translation performance of open-source autoregressive decoder-only large language models (LLMs) with task-oriented machine translation (MT) models.<n>Our experiments focus on the medical domain and cover four language directions with varied resource availability.
  arXiv  Detail & Related papers  (2024-12-08T08:54:13Z)
• Extrapolating Large Language Models to Non-English by Aligning Languages [109.09051737966178]
  Existing large language models show disparate capability across different languages. In this paper, we empower pre-trained LLMs on non-English languages by building semantic alignment across languages.
  arXiv  Detail & Related papers  (2023-08-09T13:32:06Z)
• PolyLM: An Open Source Polyglot Large Language Model [57.64420154135178]
  We present PolyLM, a multilingual large language model (LLMs) trained on 640 billion (B) tokens, avaliable in two model sizes: 1.7B and 13B. To enhance its multilingual capabilities, we 1) integrate bilingual data into training data; and 2) adopt a curriculum learning strategy that increases the proportion of non-English data from 30% in the first stage to 60% in the final stage during pre-training. Further, we propose a multilingual self-instruct method which automatically generates 132.7K diverse multilingual instructions for model fine-tuning.
  arXiv  Detail & Related papers  (2023-07-12T09:00:37Z)
• Building Machine Translation Systems for the Next Thousand Languages [102.24310122155073]
  We describe results in three research domains: building clean, web-mined datasets for 1500+ languages, developing practical MT models for under-served languages, and studying the limitations of evaluation metrics for these languages. We hope that our work provides useful insights to practitioners working towards building MT systems for currently understudied languages, and highlights research directions that can complement the weaknesses of massively multilingual models in data-sparse settings.
  arXiv  Detail & Related papers  (2022-05-09T00:24:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.