Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

• URL: http://arxiv.org/abs/2104.06951v1
• Date: Wed, 14 Apr 2021 16:21:37 GMT
• Title: Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey
• Authors: Danielle Saunders
• Abstract summary: We focus on robust approaches to domain adaptation for Neural Machine Translation (NMT) models. In particular, we look at the case where a system may need to translate sentences from multiple domains. We highlight the benefits of domain adaptation and multi-domain adaptation techniques to other lines of NMT research.
• Score: 9.645196221785694
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The development of deep learning techniques has allowed Neural Machine Translation (NMT) models to become extremely powerful, given sufficient training data and training time. However, systems struggle when translating text from a new domain with a distinct style or vocabulary. Tuning on a representative training corpus allows good in-domain translation, but such data-centric approaches can cause over-fitting to new data and `catastrophic forgetting' of previously learned behaviour. We concentrate on more robust approaches to domain adaptation for NMT, particularly the case where a system may need to translate sentences from multiple domains. We divide techniques into those relating to data selection, model architecture, parameter adaptation procedure, and inference procedure. We finally highlight the benefits of domain adaptation and multi-domain adaptation techniques to other lines of NMT research.

Related papers

• Investigating the potential of Sparse Mixtures-of-Experts for multi-domain neural machine translation [59.41178047749177]
  We focus on multi-domain Neural Machine Translation, with the goal of developing efficient models which can handle data from various domains seen during training and are robust to domains unseen during training. We hypothesize that Sparse Mixture-of-Experts (SMoE) models are a good fit for this task, as they enable efficient model scaling. We conduct a series of experiments aimed at validating the utility of SMoE for the multi-domain scenario, and find that a straightforward width scaling of Transformer is a simpler and surprisingly more efficient approach in practice, and reaches the same performance level as SMoE.
  arXiv  Detail & Related papers  (2024-07-01T09:45:22Z)
• $m^4Adapter$: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter [128.69723410769586]
  Multilingual neural machine translation models (MNMT) yield state-of-the-art performance when evaluated on data from a domain and language pair. When a MNMT model is used to translate under domain shift or to a new language pair, performance drops dramatically. We propose $m4Adapter$, which combines domain and language knowledge using meta-learning with adapters.
  arXiv  Detail & Related papers  (2022-10-21T12:25:05Z)
• DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation [10.03007605098947]
  Domain Adaptation (DA) of Neural Machine Translation (NMT) model often relies on a pre-trained general NMT model which is adapted to the new domain on a sample of in-domain parallel data. We propose a Domain Learning Curve prediction (DaLC) model that predicts prospective DA performance based on in-domain monolingual samples in the source language.
  arXiv  Detail & Related papers  (2022-04-20T06:57:48Z)
• Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation [61.27321597981737]
  $k$NN-MT has shown the promising capability of directly incorporating the pre-trained neural machine translation (NMT) model with domain-specific token-level $k$-nearest-neighbor retrieval. We propose a novel framework that directly uses in-domain monolingual sentences in the target language to construct an effective datastore for $k$-nearest-neighbor retrieval.
  arXiv  Detail & Related papers  (2021-09-14T11:50:01Z)
• Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation [19.973201669851626]
  We propose a novel meta-curriculum learning for domain adaptation in neural machine translation (NMT) During meta-training, the NMT first learns the similar curricula from each domain to avoid falling into a bad local optimum early. We show that meta-curriculum learning can improve the translation performance of both familiar and unfamiliar domains.
  arXiv  Detail & Related papers  (2021-03-03T08:58:39Z)
• Multi-Domain Adaptation in Neural Machine Translation Through Multidimensional Tagging [1.433758865948252]
  We describe and empirically evaluate multidimensional tagging (MDT), a simple yet effective method for passing sentence-level information to the model. Our human and BLEU evaluation results show that MDT can be applied to the problem of multi-domain adaptation and significantly reduce training costs without sacrificing the translation quality on any of the constituent domains.
  arXiv  Detail & Related papers  (2021-02-19T21:19:42Z)
• FDMT: A Benchmark Dataset for Fine-grained Domain Adaptation in Machine Translation [53.87731008029645]
  We present a real-world fine-grained domain adaptation task in machine translation (FDMT) The FDMT dataset consists of four sub-domains of information technology: autonomous vehicles, AI education, real-time networks and smart phone. We make quantitative experiments and deep analyses in this new setting, which benchmarks the fine-grained domain adaptation task.
  arXiv  Detail & Related papers  (2020-12-31T17:15:09Z)
• Iterative Domain-Repaired Back-Translation [50.32925322697343]
  In this paper, we focus on the domain-specific translation with low resources, where in-domain parallel corpora are scarce or nonexistent. We propose a novel iterative domain-repaired back-translation framework, which introduces the Domain-Repair model to refine translations in synthetic bilingual data. Experiments on adapting NMT models between specific domains and from the general domain to specific domains demonstrate the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2020-10-06T04:38:09Z)
• Vocabulary Adaptation for Distant Domain Adaptation in Neural Machine Translation [14.390932594872233]
  Domain adaptation between distant domains cannot be performed effectively due to mismatches in vocabulary. We propose vocabulary adaptation, a simple method for effective fine-tuning. Our method improves the performance of conventional fine-tuning by 3.86 and 3.28 BLEU points in En-Ja and De-En translation.
  arXiv  Detail & Related papers  (2020-04-30T14:27:59Z)
• A Simple Baseline to Semi-Supervised Domain Adaptation for Machine Translation [73.3550140511458]
  State-of-the-art neural machine translation (NMT) systems are data-hungry and perform poorly on new domains with no supervised data. We propose a simple but effect approach to the semi-supervised domain adaptation scenario of NMT. This approach iteratively trains a Transformer-based NMT model via three training objectives: language modeling, back-translation, and supervised translation.
  arXiv  Detail & Related papers  (2020-01-22T16:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.