DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation

• URL: http://arxiv.org/abs/2204.09259v1
• Date: Wed, 20 Apr 2022 06:57:48 GMT
• Title: DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation
• Authors: Cheonbok Park, Hantae Kim, Ioan Calapodescu, Hyunchang Cho, and Vassilina Nikoulina
• Abstract summary: Domain Adaptation (DA) of Neural Machine Translation (NMT) model often relies on a pre-trained general NMT model which is adapted to the new domain on a sample of in-domain parallel data. We propose a Domain Learning Curve prediction (DaLC) model that predicts prospective DA performance based on in-domain monolingual samples in the source language.
• Score: 10.03007605098947
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Domain Adaptation (DA) of Neural Machine Translation (NMT) model often relies on a pre-trained general NMT model which is adapted to the new domain on a sample of in-domain parallel data. Without parallel data, there is no way to estimate the potential benefit of DA, nor the amount of parallel samples it would require. It is however a desirable functionality that could help MT practitioners to make an informed decision before investing resources in dataset creation. We propose a Domain adaptation Learning Curve prediction (DaLC) model that predicts prospective DA performance based on in-domain monolingual samples in the source language. Our model relies on the NMT encoder representations combined with various instance and corpus-level features. We demonstrate that instance-level is better able to distinguish between different domains compared to corpus-level frameworks proposed in previous studies. Finally, we perform in-depth analyses of the results highlighting the limitations of our approach, and provide directions for future research.

Related papers

• LAMA-Net: Unsupervised Domain Adaptation via Latent Alignment and Manifold Learning for RUL Prediction [0.0]
  We propose textitLAMA-Net, an encoder-decoder based model (Transformer) with an induced bottleneck, Latent Alignment using Mean Maximum Discrepancy (MMD) and manifold learning. The proposed method offers a promising approach to perform domain adaptation in RUL prediction.
  arXiv  Detail & Related papers  (2022-08-17T16:28:20Z)
• Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation [50.54059385277964]
  We present a novel data augmentation paradigm termed Continuous Semantic Augmentation (CsaNMT) CsaNMT augments each training instance with an adjacency region that could cover adequate variants of literal expression under the same meaning.
  arXiv  Detail & Related papers  (2022-04-14T08:16:28Z)
• Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation [61.27321597981737]
  $k$NN-MT has shown the promising capability of directly incorporating the pre-trained neural machine translation (NMT) model with domain-specific token-level $k$-nearest-neighbor retrieval. We propose a novel framework that directly uses in-domain monolingual sentences in the target language to construct an effective datastore for $k$-nearest-neighbor retrieval.
  arXiv  Detail & Related papers  (2021-09-14T11:50:01Z)
• Efficient Nearest Neighbor Language Models [114.40866461741795]
  Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore. We show how to achieve up to a 6x speed-up in inference speed while retaining comparable performance.
  arXiv  Detail & Related papers  (2021-09-09T12:32:28Z)
• Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey [9.645196221785694]
  We focus on robust approaches to domain adaptation for Neural Machine Translation (NMT) models. In particular, we look at the case where a system may need to translate sentences from multiple domains. We highlight the benefits of domain adaptation and multi-domain adaptation techniques to other lines of NMT research.
  arXiv  Detail & Related papers  (2021-04-14T16:21:37Z)
• Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learning [27.86606560170401]
  We present a novel meta-learning algorithm for unsupervised neural machine translation (UNMT) We train the model to adapt to another domain by utilizing only a small amount of training data. Our model surpasses a transfer learning-based approach by up to 2-4 BLEU scores.
  arXiv  Detail & Related papers  (2020-10-18T17:54:13Z)
• Iterative Domain-Repaired Back-Translation [50.32925322697343]
  In this paper, we focus on the domain-specific translation with low resources, where in-domain parallel corpora are scarce or nonexistent. We propose a novel iterative domain-repaired back-translation framework, which introduces the Domain-Repair model to refine translations in synthetic bilingual data. Experiments on adapting NMT models between specific domains and from the general domain to specific domains demonstrate the effectiveness of our proposed approach.
  arXiv  Detail & Related papers  (2020-10-06T04:38:09Z)
• Learning Contextualized Sentence Representations for Document-Level Neural Machine Translation [59.191079800436114]
  Document-level machine translation incorporates inter-sentential dependencies into the translation of a source sentence. We propose a new framework to model cross-sentence dependencies by training neural machine translation (NMT) to predict both the target translation and surrounding sentences of a source sentence.
  arXiv  Detail & Related papers  (2020-03-30T03:38:01Z)
• A Simple Baseline to Semi-Supervised Domain Adaptation for Machine Translation [73.3550140511458]
  State-of-the-art neural machine translation (NMT) systems are data-hungry and perform poorly on new domains with no supervised data. We propose a simple but effect approach to the semi-supervised domain adaptation scenario of NMT. This approach iteratively trains a Transformer-based NMT model via three training objectives: language modeling, back-translation, and supervised translation.
  arXiv  Detail & Related papers  (2020-01-22T16:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.