Efficient Machine Translation Domain Adaptation
- URL: http://arxiv.org/abs/2204.12608v1
- Date: Tue, 26 Apr 2022 21:47:54 GMT
- Title: Efficient Machine Translation Domain Adaptation
- Authors: Pedro Henrique Martins and Zita Marinho and Andr\'e F. T. Martins
- Abstract summary: Machine translation models struggle when translating out-of-domain text.
domain adaptation methods focus on fine-tuning or training the entire or part of the model on every new domain.
We introduce a simple but effective caching strategy that avoids performing retrieval when similar contexts have been seen before.
- Score: 7.747003493657217
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Machine translation models struggle when translating out-of-domain text,
which makes domain adaptation a topic of critical importance. However, most
domain adaptation methods focus on fine-tuning or training the entire or part
of the model on every new domain, which can be costly. On the other hand,
semi-parametric models have been shown to successfully perform domain
adaptation by retrieving examples from an in-domain datastore (Khandelwal et
al., 2021). A drawback of these retrieval-augmented models, however, is that
they tend to be substantially slower. In this paper, we explore several
approaches to speed up nearest neighbor machine translation. We adapt the
methods recently proposed by He et al. (2021) for language modeling, and
introduce a simple but effective caching strategy that avoids performing
retrieval when similar contexts have been seen before. Translation quality and
runtimes for several domains show the effectiveness of the proposed solutions.
Related papers
- Efficient Hierarchical Domain Adaptation for Pretrained Language Models [77.02962815423658]
Generative language models are trained on diverse, general domain corpora.
We introduce a method to scale domain adaptation to many diverse domains using a computationally efficient adapter approach.
arXiv Detail & Related papers (2021-12-16T11:09:29Z) - Domain Adaptation and Multi-Domain Adaptation for Neural Machine
Translation: A Survey [9.645196221785694]
We focus on robust approaches to domain adaptation for Neural Machine Translation (NMT) models.
In particular, we look at the case where a system may need to translate sentences from multiple domains.
We highlight the benefits of domain adaptation and multi-domain adaptation techniques to other lines of NMT research.
arXiv Detail & Related papers (2021-04-14T16:21:37Z) - Pruning-then-Expanding Model for Domain Adaptation of Neural Machine
Translation [9.403585397617865]
Domain adaptation is widely used in practical applications of neural machine translation.
The existing methods for domain adaptation usually suffer from catastrophic forgetting, domain divergence, and model explosion.
We propose a method of "divide and conquer" which is based on the importance of neurons or parameters in the translation model.
arXiv Detail & Related papers (2021-03-25T08:57:09Z) - Model-Based Domain Generalization [96.84818110323518]
We propose a novel approach for the domain generalization problem called Model-Based Domain Generalization.
Our algorithms beat the current state-of-the-art methods on the very-recently-proposed WILDS benchmark by up to 20 percentage points.
arXiv Detail & Related papers (2021-02-23T00:59:02Z) - Rapid Domain Adaptation for Machine Translation with Monolingual Data [31.70276147485463]
One challenge of machine translation is how to quickly adapt to unseen domains in face of surging events like COVID-19.
In this paper, we propose an approach that enables rapid domain adaptation from the perspective of unsupervised translation.
arXiv Detail & Related papers (2020-10-23T20:31:37Z) - Iterative Domain-Repaired Back-Translation [50.32925322697343]
In this paper, we focus on the domain-specific translation with low resources, where in-domain parallel corpora are scarce or nonexistent.
We propose a novel iterative domain-repaired back-translation framework, which introduces the Domain-Repair model to refine translations in synthetic bilingual data.
Experiments on adapting NMT models between specific domains and from the general domain to specific domains demonstrate the effectiveness of our proposed approach.
arXiv Detail & Related papers (2020-10-06T04:38:09Z) - Domain Adaptation for Semantic Parsing [68.81787666086554]
We propose a novel semantic for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain.
Our semantic benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages.
Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies.
arXiv Detail & Related papers (2020-06-23T14:47:41Z) - Supervised Domain Adaptation using Graph Embedding [86.3361797111839]
Domain adaptation methods assume that distributions between the two domains are shifted and attempt to realign them.
We propose a generic framework based on graph embedding.
We show that the proposed approach leads to a powerful Domain Adaptation framework.
arXiv Detail & Related papers (2020-03-09T12:25:13Z) - Learning to adapt class-specific features across domains for semantic
segmentation [36.36210909649728]
In this thesis, we present a novel architecture, which learns to adapt features across domains by taking into account per class information.
We adopt the recently introduced StarGAN architecture as image translation backbone, since it is able to perform translations across multiple domains by means of a single generator network.
arXiv Detail & Related papers (2020-01-22T23:51:30Z) - A Simple Baseline to Semi-Supervised Domain Adaptation for Machine
Translation [73.3550140511458]
State-of-the-art neural machine translation (NMT) systems are data-hungry and perform poorly on new domains with no supervised data.
We propose a simple but effect approach to the semi-supervised domain adaptation scenario of NMT.
This approach iteratively trains a Transformer-based NMT model via three training objectives: language modeling, back-translation, and supervised translation.
arXiv Detail & Related papers (2020-01-22T16:42:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.