Translating Similar Languages: Role of Mutual Intelligibility in Multilingual Transformers

• URL: http://arxiv.org/abs/2011.05037v1
• Date: Tue, 10 Nov 2020 10:58:38 GMT
• Title: Translating Similar Languages: Role of Mutual Intelligibility in Multilingual Transformers
• Authors: Ife Adebara, El Moatez Billah Nagoudi, Muhammad Abdul Mageed
• Abstract summary: We investigate approaches to translate between similar languages under low resource conditions. We submit Transformer-based bilingual and multilingual systems for all language pairs. Our Spanish-Catalan model has the best performance of all the five language pairs.
• Score: 8.9379057739817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate different approaches to translate between similar languages under low resource conditions, as part of our contribution to the WMT 2020 Similar Languages Translation Shared Task. We submitted Transformer-based bilingual and multilingual systems for all language pairs, in the two directions. We also leverage back-translation for one of the language pairs, acquiring an improvement of more than 3 BLEU points. We interpret our results in light of the degree of mutual intelligibility (based on Jaccard similarity) between each pair, finding a positive correlation between mutual intelligibility and model performance. Our Spanish-Catalan model has the best performance of all the five language pairs. Except for the case of Hindi-Marathi, our bilingual models achieve better performance than the multilingual models on all pairs.

Related papers

• Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation [129.99918589405675]
  Large multilingual models trained with self-supervision achieve state-of-the-art results in a wide range of natural language processing tasks. Multilingual fine-tuning improves performance on low-resource languages but requires modifying the entire model and can be prohibitively expensive. We propose training language-family adapters on top of mBART-50 to facilitate cross-lingual transfer.
  arXiv  Detail & Related papers  (2022-09-30T05:02:42Z)
• Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer [14.799109368073548]
  We exploit the pre-trained seq2seq model mBART for multilingual text style transfer. Using machine translated data as well as gold aligned English sentences yields state-of-the-art results.
  arXiv  Detail & Related papers  (2022-03-16T11:27:48Z)
• Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation [45.77509642452541]
  We introduce multilingual crossover encoder-decoder (mXEncDec) to fuse language pairs at an instance level. Our approach interpolates instances from different language pairs into joint crossover examples' in order to encourage sharing input and output spaces across languages.
  arXiv  Detail & Related papers  (2022-03-15T03:56:22Z)
• CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task [0.0]
  We show that using joint model for multiple similar language pairs improves upon translation quality in each pair. We also demonstrate that chararacter-level bilingual models are competitive for very similar language pairs.
  arXiv  Detail & Related papers  (2021-09-20T08:10:39Z)
• Establishing Interlingua in Multilingual Language Models [0.0]
  We show that different languages do converge to a shared space in large multilingual language models. We extend our analysis to 28 diverse languages and find that the interlingual space exhibits a particular structure similar to the linguistic relatedness of languages.
  arXiv  Detail & Related papers  (2021-09-02T20:53:14Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)
• Are Multilingual Models Effective in Code-Switching? [57.78477547424949]
  We study the effectiveness of multilingual language models to understand their capability and adaptability to the mixed-language setting. Our findings suggest that pre-trained multilingual models do not necessarily guarantee high-quality representations on code-switching.
  arXiv  Detail & Related papers  (2021-03-24T16:20:02Z)
• Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation [105.41167108465085]
  Cross-lingual Machine Reading (CLMRC) remains a challenging problem due to the lack of large-scale datasets in low-source languages. We propose a novel augmentation approach named Language Branch Machine Reading (LBMRC) LBMRC trains multiple machine reading comprehension (MRC) models proficient in individual language. We devise a multilingual distillation approach to amalgamate knowledge from multiple language branch models to a single model for all target languages.
  arXiv  Detail & Related papers  (2020-10-27T13:12:17Z)
• Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations [83.27475281544868]
  We use singular vector canonical correlation analysis to study what kind of information is induced from each source. We observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy.
  arXiv  Detail & Related papers  (2020-04-30T16:25:39Z)
• Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation [61.88012735215636]
  Unsupervised neural machine translation (UNMT) has recently achieved remarkable results for several language pairs. UNMT can only translate between a single language pair and cannot produce translation results for multiple language pairs at the same time. In this paper, we empirically introduce a simple method to translate between thirteen languages using a single encoder and a single decoder.
  arXiv  Detail & Related papers  (2020-04-21T17:26:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.