From Priest to Doctor: Domain Adaptaion for Low-Resource Neural Machine Translation

• URL: http://arxiv.org/abs/2412.00966v2
• Date: Thu, 30 Jan 2025 20:13:25 GMT
• Title: From Priest to Doctor: Domain Adaptaion for Low-Resource Neural Machine Translation
• Authors: Ali Marashian, Enora Rice, Luke Gessler, Alexis Palmer, Katharina von der Wense,
• Abstract summary: Many languages have insufficient data to train high-performing general neural machine translation (NMT) models. Many of the world's languages have insufficient data to train high-performing general neural machine translation (NMT) models.
• Score: 3.666125285899499
• License:
• Abstract: Many of the world's languages have insufficient data to train high-performing general neural machine translation (NMT) models, let alone domain-specific models, and often the only available parallel data are small amounts of religious texts. Hence, domain adaptation (DA) is a crucial issue faced by contemporary NMT and has, so far, been underexplored for low-resource languages. In this paper, we evaluate a set of methods from both low-resource NMT and DA in a realistic setting, in which we aim to translate between a high-resource and a low-resource language with access to only: a) parallel Bible data, b) a bilingual dictionary, and c) a monolingual target-domain corpus in the high-resource language. Our results show that the effectiveness of the tested methods varies, with the simplest one, DALI, being most effective. We follow up with a small human evaluation of DALI, which shows that there is still a need for more careful investigation of how to accomplish DA for low-resource NMT.

Related papers

• Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu [53.437954702561065]
  In-context machine translation (MT) with large language models (LLMs) is a promising approach for low-resource MT. This study systematically investigates how each resource and its quality affects the translation performance, with the Manchu language. Our results indicate that high-quality dictionaries and good parallel examples are very helpful, while grammars hardly help.
  arXiv  Detail & Related papers  (2025-02-17T14:53:49Z)
• Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation [62.202893186343935]
  We explore what it would take to adapt Large Language Models for low-resource languages. We show that parallel data is critical during both pre-training andSupervised Fine-Tuning (SFT) Our experiments with three LLMs across two low-resourced language groups reveal consistent trends, underscoring the generalizability of our findings.
  arXiv  Detail & Related papers  (2024-08-23T00:59:38Z)
• When Does Monolingual Data Help Multilingual Translation: The Role of Domain and Model Scale [73.69252847606212]
  We examine how denoising autoencoding (DAE) and backtranslation (BT) impact machine translation (MMT) We find that monolingual data generally helps MMT, but models are surprisingly brittle to domain mismatches, especially at smaller model scales. As scale increases, DAE transitions from underperforming the parallel-only baseline at 90M to converging with BT performance at 1.6B, and even surpassing it in low-resource.
  arXiv  Detail & Related papers  (2023-05-23T14:48:42Z)
• Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation [103.90963418039473]
  Bi-ACL is a framework that uses only target-side monolingual data and a bilingual dictionary to improve the performance of the MNMT model. We show that Bi-ACL is more effective both in long-tail languages and in high-resource languages.
  arXiv  Detail & Related papers  (2023-05-22T07:31:08Z)
• Low-resource Neural Machine Translation with Cross-modal Alignment [15.416659725808822]
  We propose a cross-modal contrastive learning method to learn a shared space for all languages. Experimental results and further analysis show that our method can effectively learn the cross-modal and cross-lingual alignment with a small amount of image-text pairs.
  arXiv  Detail & Related papers  (2022-10-13T04:15:43Z)
• AUGVIC: Exploiting BiText Vicinity for Low-Resource NMT [9.797319790710711]
  AUGVIC is a novel data augmentation framework for low-resource NMT. It exploits the vicinal samples of the given bitext without using any extra monolingual data explicitly. We show that AUGVIC helps to attenuate the discrepancies between relevant and distant-domain monolingual data in traditional back-translation.
  arXiv  Detail & Related papers  (2021-06-09T15:29:18Z)
• Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data [40.11208706647032]
  The scarcity of parallel data is a major obstacle for training high-quality machine translation systems for low-resource languages. In this work, we exploit this linguistic overlap to facilitate translating to and from a low-resource language with only monolingual data. Our method, NMT-Adapt, combines denoising autoencoding, back-translation and adversarial objectives to utilize monolingual data for low-resource adaptation.
  arXiv  Detail & Related papers  (2021-05-31T16:01:18Z)
• How Low is Too Low? A Computational Perspective on Extremely Low-Resource Languages [1.7625363344837164]
  We introduce the first cross-lingual information extraction pipeline for Sumerian. We also curate InterpretLR, an interpretability toolkit for low-resource NLP. Most components of our pipeline can be generalised to any other language to obtain an interpretable execution.
  arXiv  Detail & Related papers  (2021-05-30T12:09:59Z)
• A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages [14.694800341598368]
  We focus on dependency parsing for morphological rich languages (MRLs) in a low-resource setting. To address these challenges, we propose simple auxiliary tasks for pretraining. We perform experiments on 10 MRLs in low-resource settings to measure the efficacy of our proposed pretraining method.
  arXiv  Detail & Related papers  (2021-02-12T14:26:58Z)
• Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information [72.2412707779571]
  mRASP is an approach to pre-train a universal multilingual neural machine translation model. We carry out experiments on 42 translation directions across a diverse setting, including low, medium, rich resource, and as well as transferring to exotic language pairs.
  arXiv  Detail & Related papers  (2020-10-07T03:57:54Z)
• Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation [54.52971020087777]
  Using monolingual data significantly boosts the translation quality of low-resource languages in multilingual models. Self-supervision improves zero-shot translation quality in multilingual models. We get up to 33 BLEU on ro-en translation without any parallel data or back-translation.
  arXiv  Detail & Related papers  (2020-05-11T00:20:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.