Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation

• URL: http://arxiv.org/abs/2103.02262v1
• Date: Wed, 3 Mar 2021 08:58:39 GMT
• Title: Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation
• Authors: Runzhe Zhan, Xuebo Liu, Derek F. Wong, Lidia S. Chao
• Abstract summary: We propose a novel meta-curriculum learning for domain adaptation in neural machine translation (NMT) During meta-training, the NMT first learns the similar curricula from each domain to avoid falling into a bad local optimum early. We show that meta-curriculum learning can improve the translation performance of both familiar and unfamiliar domains.
• Score: 19.973201669851626
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Meta-learning has been sufficiently validated to be beneficial for low-resource neural machine translation (NMT). However, we find that meta-trained NMT fails to improve the translation performance of the domain unseen at the meta-training stage. In this paper, we aim to alleviate this issue by proposing a novel meta-curriculum learning for domain adaptation in NMT. During meta-training, the NMT first learns the similar curricula from each domain to avoid falling into a bad local optimum early, and finally learns the curricula of individualities to improve the model robustness for learning domain-specific knowledge. Experimental results on 10 different low-resource domains show that meta-curriculum learning can improve the translation performance of both familiar and unfamiliar domains. All the codes and data are freely available at https://github.com/NLP2CT/Meta-Curriculum.

Related papers

• Code-Switching with Word Senses for Pretraining in Neural Machine Translation [107.23743153715799]
  We introduce Word Sense Pretraining for Neural Machine Translation (WSP-NMT) WSP-NMT is an end-to-end approach for pretraining multilingual NMT models leveraging word sense-specific information from Knowledge Bases. Our experiments show significant improvements in overall translation quality.
  arXiv  Detail & Related papers  (2023-10-21T16:13:01Z)
• Domain Adaptation for Arabic Machine Translation: The Case of Financial Texts [0.7673339435080445]
  We develop a parallel corpus for Arabic-English (AR- EN) translation in the financial domain. We fine-tune several NMT and Large Language models including ChatGPT-3.5 Turbo. The quality of ChatGPT translation was superior than other models based on automatic and human evaluations.
  arXiv  Detail & Related papers  (2023-09-22T13:37:19Z)
• $m^4Adapter$: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter [128.69723410769586]
  Multilingual neural machine translation models (MNMT) yield state-of-the-art performance when evaluated on data from a domain and language pair. When a MNMT model is used to translate under domain shift or to a new language pair, performance drops dramatically. We propose $m4Adapter$, which combines domain and language knowledge using meta-learning with adapters.
  arXiv  Detail & Related papers  (2022-10-21T12:25:05Z)
• Can Domains Be Transferred Across Languages in Multi-Domain Multilingual Neural Machine Translation? [52.27798071809941]
  This paper investigates whether the domain information can be transferred across languages on the composition of multi-domain and multilingual NMT. We find that multi-domain multilingual (MDML) NMT can boost zero-shot translation performance up to +10 gains on BLEU.
  arXiv  Detail & Related papers  (2022-10-20T23:13:54Z)
• Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation [61.27321597981737]
  $k$NN-MT has shown the promising capability of directly incorporating the pre-trained neural machine translation (NMT) model with domain-specific token-level $k$-nearest-neighbor retrieval. We propose a novel framework that directly uses in-domain monolingual sentences in the target language to construct an effective datastore for $k$-nearest-neighbor retrieval.
  arXiv  Detail & Related papers  (2021-09-14T11:50:01Z)
• Phrase-level Active Learning for Neural Machine Translation [107.28450614074002]
  We propose an active learning setting where we can spend a given budget on translating in-domain data. We select both full sentences and individual phrases from unlabelled data in the new domain for routing to human translators. In a German-English translation task, our active learning approach achieves consistent improvements over uncertainty-based sentence selection methods.
  arXiv  Detail & Related papers  (2021-06-21T19:20:42Z)
• Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey [9.645196221785694]
  We focus on robust approaches to domain adaptation for Neural Machine Translation (NMT) models. In particular, we look at the case where a system may need to translate sentences from multiple domains. We highlight the benefits of domain adaptation and multi-domain adaptation techniques to other lines of NMT research.
  arXiv  Detail & Related papers  (2021-04-14T16:21:37Z)
• Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning [27.86606560170401]
  We present a novel meta-learning algorithm for unsupervised neural machine translation (UNMT) We train the model to adapt to another domain by utilizing only a small amount of training data. Our model surpasses a transfer learning-based approach by up to 2-4 BLEU scores.
  arXiv  Detail & Related papers  (2020-10-18T17:54:13Z)
• Vocabulary Adaptation for Distant Domain Adaptation in Neural Machine Translation [14.390932594872233]
  Domain adaptation between distant domains cannot be performed effectively due to mismatches in vocabulary. We propose vocabulary adaptation, a simple method for effective fine-tuning. Our method improves the performance of conventional fine-tuning by 3.86 and 3.28 BLEU points in En-Ja and De-En translation.
  arXiv  Detail & Related papers  (2020-04-30T14:27:59Z)
• Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining [37.2106265998237]
  We propose an effective learning procedure named Meta Fine-Tuning (MFT) MFT serves as a meta-learner to solve a group of similar NLP tasks for neural language models. We implement MFT upon BERT to solve several multi-domain text mining tasks.
  arXiv  Detail & Related papers  (2020-03-29T11:27:10Z)
• A Simple Baseline to Semi-Supervised Domain Adaptation for Machine Translation [73.3550140511458]
  State-of-the-art neural machine translation (NMT) systems are data-hungry and perform poorly on new domains with no supervised data. We propose a simple but effect approach to the semi-supervised domain adaptation scenario of NMT. This approach iteratively trains a Transformer-based NMT model via three training objectives: language modeling, back-translation, and supervised translation.
  arXiv  Detail & Related papers  (2020-01-22T16:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.