CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task

• URL: http://arxiv.org/abs/2109.09354v1
• Date: Mon, 20 Sep 2021 08:10:39 GMT
• Title: CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task
• Authors: Josef Jon, Michal Nov\'ak, Jo\~ao Paulo Aires, Du\v{s}an Vari\v{s} and Ond\v{r}ej Bojar
• Abstract summary: We show that using joint model for multiple similar language pairs improves upon translation quality in each pair. We also demonstrate that chararacter-level bilingual models are competitive for very similar language pairs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper describes Charles University submission for Multilingual Low-Resource Translation for Indo-European Languages shared task at WMT21. We competed in translation from Catalan into Romanian, Italian and Occitan. Our systems are based on shared multilingual model. We show that using joint model for multiple similar language pairs improves upon translation quality in each pair. We also demonstrate that chararacter-level bilingual models are competitive for very similar language pairs (Catalan-Occitan) but less so for more distant pairs. We also describe our experiments with multi-task learning, where aside from a textual translation, the models are also trained to perform grapheme-to-phoneme conversion.

Related papers

• Enhancing Translation for Indigenous Languages: Experiments with Multilingual Models [57.10972566048735]
  We present the system descriptions for three methods. We used two multilingual models, namely M2M-100 and mBART50, and one bilingual (one-to-one) -- Helsinki NLP Spanish-English translation model. We experimented with 11 languages from America and report the setups we used as well as the results we achieved.
  arXiv  Detail & Related papers  (2023-05-27T08:10:40Z)
• Improving Multilingual Neural Machine Translation System for Indic Languages [0.0]
  We propose a multilingual neural machine translation (MNMT) system to address the issues related to low-resource language translation. A state-of-the-art transformer architecture is used to realize the proposed model. Trials over a good amount of data reveal its superiority over the conventional models.
  arXiv  Detail & Related papers  (2022-09-27T09:51:56Z)
• Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning [48.15259834021655]
  We present a pragmatic approach towards building a multilingual machine translation model that covers hundreds of languages. We use a mixture of supervised and self-supervised objectives, depending on the data availability for different language pairs. We demonstrate that the synergy between these two training paradigms enables the model to produce high-quality translations in the zero-resource setting.
  arXiv  Detail & Related papers  (2022-01-09T23:36:44Z)
• UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training [52.852163987208826]
  UC2 is the first machine translation-augmented framework for cross-lingual cross-modal representation learning. We propose two novel pre-training tasks, namely Masked Region-to-Token Modeling (MRTM) and Visual Translation Language Modeling (VTLM) Our proposed framework achieves new state-of-the-art on diverse non-English benchmarks while maintaining comparable performance to monolingual pre-trained models on English tasks.
  arXiv  Detail & Related papers  (2021-04-01T08:30:53Z)
• Improving Multilingual Neural Machine Translation For Low-Resource Languages: French-, English- Vietnamese [4.103253352106816]
  This paper proposes two simple strategies to address the rare word issue in multilingual MT systems for two low-resource language pairs: French-Vietnamese and English-Vietnamese. We have shown significant improvements of up to +1.62 and +2.54 BLEU points over the bilingual baseline systems for both language pairs.
  arXiv  Detail & Related papers  (2020-12-16T04:43:43Z)
• Translating Similar Languages: Role of Mutual Intelligibility in Multilingual Transformers [8.9379057739817]
  We investigate approaches to translate between similar languages under low resource conditions. We submit Transformer-based bilingual and multilingual systems for all language pairs. Our Spanish-Catalan model has the best performance of all the five language pairs.
  arXiv  Detail & Related papers  (2020-11-10T10:58:38Z)
• Learning Contextualised Cross-lingual Word Embeddings and Alignments for Extremely Low-Resource Languages Using Parallel Corpora [63.5286019659504]
  We propose a new approach for learning contextualised cross-lingual word embeddings based on a small parallel corpus. Our method obtains word embeddings via an LSTM encoder-decoder model that simultaneously translates and reconstructs an input sentence.
  arXiv  Detail & Related papers  (2020-10-27T22:24:01Z)
• Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation [105.41167108465085]
  Cross-lingual Machine Reading (CLMRC) remains a challenging problem due to the lack of large-scale datasets in low-source languages. We propose a novel augmentation approach named Language Branch Machine Reading (LBMRC) LBMRC trains multiple machine reading comprehension (MRC) models proficient in individual language. We devise a multilingual distillation approach to amalgamate knowledge from multiple language branch models to a single model for all target languages.
  arXiv  Detail & Related papers  (2020-10-27T13:12:17Z)
• Beyond English-Centric Multilingual Machine Translation [74.21727842163068]
  We create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Our focus on non-English-Centric models brings gains of more than 10 BLEU when directly translating between non-English directions while performing competitively to the best single systems of WMT.
  arXiv  Detail & Related papers  (2020-10-21T17:01:23Z)
• Complete Multilingual Neural Machine Translation [44.98358050355681]
  We study the use of multi-way aligned examples to enrich the original English-centric parallel corpora. We call MNMT with such connectivity pattern complete Multilingual Neural Machine Translation (cMNMT) In combination with a novel training data sampling strategy that is conditioned on the target language only, cMNMT yields competitive translation quality for all language pairs.
  arXiv  Detail & Related papers  (2020-10-20T13:03:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.