An Empirical Investigation of Multi-bridge Multilingual NMT models

• URL: http://arxiv.org/abs/2110.07304v1
• Date: Thu, 14 Oct 2021 12:14:22 GMT
• Title: An Empirical Investigation of Multi-bridge Multilingual NMT models
• Abstract summary: We present an investigation of multi-bridge, many-to-many multilingual NMT models (MB-M2M) Models trained on non-English language pairs and English-centric language pairs.
• Score: 4.449118708319941
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we present an extensive investigation of multi-bridge, many-to-many multilingual NMT models (MB-M2M) ie., models trained on non-English language pairs in addition to English-centric language pairs. In addition to validating previous work which shows that MB-M2M models can overcome zeroshot translation problems, our analysis reveals the following results about multibridge models: (1) it is possible to extract a reasonable amount of parallel corpora between non-English languages for low-resource languages (2) with limited non-English centric data, MB-M2M models are competitive with or outperform pivot models, (3) MB-M2M models can outperform English-Any models and perform at par with Any-English models, so a single multilingual NMT system can serve all translation directions.

Related papers

• CAT-Translate: Building Compact Open-Source Models for Japanese-English Translation [16.295305195753723]
  We develop a family of small language models (0.8B, 1.4B, 3.3B, and 7B parameters) specialized for Japanese-English bidirectional translation.<n>We evaluate our models on WMT and real-world translation benchmarks across business, legal, medical, financial, and patent domains.
  arXiv  Detail & Related papers  (2026-06-19T13:28:59Z)
• Omnilingual MT: Machine Translation for 1,600 Languages [58.66170104105936]
  We present Omnilingual Machine Translation (OMT), the first machine translation system supporting more than 1,600 languages.<n>This scale is enabled by a comprehensive data strategy that integrates large public multilingual corpora with newly created datasets.<n>OMT models improve in cross-lingual transfer, being close to solving the "understanding" part of the puzzle in MT for the 1,600 evaluated.
  arXiv  Detail & Related papers  (2026-03-17T09:43:42Z)
• Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters [53.59868121093848]
  We introduce Seed-X, a family of open-source language models (LLMs) with 7B parameter size.<n>The base model is pre-trained on a diverse, high-quality dataset encompassing both monolingual and bilingual content across 28 languages.<n>The instruct model is then finetuned to translate by Chain-of-Thought (CoT) reasoning and further enhanced through reinforcement learning (RL) to achieve better generalization across diverse language pairs.
  arXiv  Detail & Related papers  (2025-07-18T03:19:43Z)
• MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining [27.952041404675846]
  We introduce MuRating, a framework that transfers high-quality English data-quality signals into a single rater for 17 target languages.<n>MuRating aggregates multiple English "raters" via pairwise comparisons to learn unified document-quality scores.<n>It then projects these judgments through translation to train a multilingual evaluator on monolingual, cross-lingual, and parallel text pairs.
  arXiv  Detail & Related papers  (2025-07-02T15:11:12Z)
• M-Prometheus: A Suite of Open Multilingual LLM Judges [64.22940792713713]
  We introduce M-Prometheus, a suite of open-weight LLM judges that can provide both direct assessment and pairwise comparison feedback on multilingual outputs. M-Prometheus models outperform state-of-the-art open LLM judges on multilingual reward benchmarks spanning more than 20 languages, as well as on literary machine translation (MT) evaluation covering 4 language pairs.
  arXiv  Detail & Related papers  (2025-04-07T11:37:26Z)
• Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study [13.409987421121405]
  GemmaX2-28 is a 9B model achieving top-tier multilingual translation performance across 28 languages. GemmaX2-28 consistently outperforms the state-of-the-art (SOTA) models such as TowerInstruct and XALMA.
  arXiv  Detail & Related papers  (2025-02-04T16:57:03Z)
• YAYI 2: Multilingual Open-Source Large Language Models [53.92832054643197]
  We propose YAYI 2, including both base and chat models, with 30 billion parameters. YAYI 2 is pre-trained from scratch on a multilingual corpus which contains 2.65 trillion tokens filtered by our pre-training data processing pipeline. The base model is aligned with human values through supervised fine-tuning with millions of instructions and reinforcement learning from human feedback.
  arXiv  Detail & Related papers  (2023-12-22T17:34:47Z)
• PolyLM: An Open Source Polyglot Large Language Model [57.64420154135178]
  We present PolyLM, a multilingual large language model (LLMs) trained on 640 billion (B) tokens, avaliable in two model sizes: 1.7B and 13B. To enhance its multilingual capabilities, we 1) integrate bilingual data into training data; and 2) adopt a curriculum learning strategy that increases the proportion of non-English data from 30% in the first stage to 60% in the final stage during pre-training. Further, we propose a multilingual self-instruct method which automatically generates 132.7K diverse multilingual instructions for model fine-tuning.
  arXiv  Detail & Related papers  (2023-07-12T09:00:37Z)
• Distilling Efficient Language-Specific Models for Cross-Lingual Transfer [75.32131584449786]
  Massively multilingual Transformers (MMTs) are widely used for cross-lingual transfer learning. MMTs' language coverage makes them unnecessarily expensive to deploy in terms of model size, inference time, energy, and hardware cost. We propose to extract compressed, language-specific models from MMTs which retain the capacity of the original MMTs for cross-lingual transfer.
  arXiv  Detail & Related papers  (2023-06-02T17:31:52Z)
• Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation [48.37939354609931]
  We propose a novel efficient training recipe, upon which we build an effective detachable model, Lego-MT. Experiments show that Lego-MT with 1.2B parameters brings an average gain of 3.2 spBLEU. The proposed training recipe brings a 28.2$times$ speedup over the conventional multi-way training method.
  arXiv  Detail & Related papers  (2022-12-20T18:54:08Z)
• MiLMo:Minority Multilingual Pre-trained Language Model [1.6409017540235764]
  This paper constructs a multilingual pre-trained model named MiLMo that performs better on minority language tasks. By comparing the word2vec model and the pre-trained model in the text classification task, this paper provides an optimal scheme for the downstream task research of minority languages.
  arXiv  Detail & Related papers  (2022-12-04T09:28:17Z)
• Building Machine Translation Systems for the Next Thousand Languages [102.24310122155073]
  We describe results in three research domains: building clean, web-mined datasets for 1500+ languages, developing practical MT models for under-served languages, and studying the limitations of evaluation metrics for these languages. We hope that our work provides useful insights to practitioners working towards building MT systems for currently understudied languages, and highlights research directions that can complement the weaknesses of massively multilingual models in data-sparse settings.
  arXiv  Detail & Related papers  (2022-05-09T00:24:13Z)
• Cross-Lingual Text Classification with Multilingual Distillation and Zero-Shot-Aware Training [21.934439663979663]
  Multi-branch multilingual language model (MBLM) built on Multilingual pre-trained language models (MPLMs) Method based on transferring knowledge from high-performance monolingual models with a teacher-student framework. Results on two cross-lingual classification tasks show that, with only the task's supervised data used, our method improves both the supervised and zero-shot performance of MPLMs.
  arXiv  Detail & Related papers  (2022-02-28T09:51:32Z)
• Beyond English-Centric Multilingual Machine Translation [74.21727842163068]
  We create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Our focus on non-English-Centric models brings gains of more than 10 BLEU when directly translating between non-English directions while performing competitively to the best single systems of WMT.
  arXiv  Detail & Related papers  (2020-10-21T17:01:23Z)
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation [81.7786241489002]
  Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics. We propose random online backtranslation to enforce the translation of unseen training language pairs.
  arXiv  Detail & Related papers  (2020-04-24T17:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.