Negative Lexical Constraints in Neural Machine Translation

• URL: http://arxiv.org/abs/2308.03601v1
• Date: Mon, 7 Aug 2023 14:04:15 GMT
• Title: Negative Lexical Constraints in Neural Machine Translation
• Authors: Josef Jon, Du\v{s}an Vari\v{s}, Michal Nov\'ak, Jo\~ao Paulo Aires and Ond\v{r}ej Bojar
• Abstract summary: Negative lexical constraining is used to prohibit certain words or expressions in the translation produced by the neural translation model. We compare various methods based on modifying either the decoding process or the training data. We demonstrate that our method improves the constraining, although the problem still persists in many cases.
• Score: 1.3124513975412255
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper explores negative lexical constraining in English to Czech neural machine translation. Negative lexical constraining is used to prohibit certain words or expressions in the translation produced by the neural translation model. We compared various methods based on modifying either the decoding process or the training data. The comparison was performed on two tasks: paraphrasing and feedback-based translation refinement. We also studied to which extent these methods "evade" the constraints presented to the model (usually in the dictionary form) by generating a different surface form of a given constraint.We propose a way to mitigate the issue through training with stemmed negative constraints to counter the model's ability to induce a variety of the surface forms of a word that can result in bypassing the constraint. We demonstrate that our method improves the constraining, although the problem still persists in many cases.

Related papers

• An Analysis of BPE Vocabulary Trimming in Neural Machine Translation [56.383793805299234]
  vocabulary trimming is a postprocessing step that replaces rare subwords with their component subwords. We show that vocabulary trimming fails to improve performance and is even prone to incurring heavy degradation.
  arXiv  Detail & Related papers  (2024-03-30T15:29:49Z)
• Pixel Sentence Representation Learning [67.4775296225521]
  In this work, we conceptualize the learning of sentence-level textual semantics as a visual representation learning process. We employ visually-grounded text perturbation methods like typos and word order shuffling, resonating with human cognitive patterns, and enabling perturbation to be perceived as continuous. Our approach is further bolstered by large-scale unsupervised topical alignment training and natural language inference supervision.
  arXiv  Detail & Related papers  (2024-02-13T02:46:45Z)
• A Template-based Method for Constrained Neural Machine Translation [100.02590022551718]
  We propose a template-based method that can yield results with high translation quality and match accuracy while keeping the decoding speed. The generation and derivation of the template can be learned through one sequence-to-sequence training framework. Experimental results show that the proposed template-based methods can outperform several representative baselines in lexically and structurally constrained translation tasks.
  arXiv  Detail & Related papers  (2022-05-23T12:24:34Z)
• Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategies [72.56158036639707]
  Morphologically rich languages pose difficulties to machine translation. A large amount of differently inflected word surface forms entails a larger vocabulary. Some inflected forms of infrequent terms typically do not appear in the training corpus. Linguistic agreement requires the system to correctly match the grammatical categories between inflected word forms in the output sentence.
  arXiv  Detail & Related papers  (2022-03-25T10:13:20Z)
• DEEP: DEnoising Entity Pre-training for Neural Machine Translation [123.6686940355937]
  It has been shown that machine translation models usually generate poor translations for named entities that are infrequent in the training corpus. We propose DEEP, a DEnoising Entity Pre-training method that leverages large amounts of monolingual data and a knowledge base to improve named entity translation accuracy within sentences.
  arXiv  Detail & Related papers  (2021-11-14T17:28:09Z)
• End-to-End Lexically Constrained Machine Translation for Morphologically Rich Languages [0.0]
  We investigate mechanisms to allow neural machine translation to infer the correct word inflection given lemmatized constraints. Our experiments on the English-Czech language pair show that this approach improves the translation of constrained terms in both automatic and manual evaluation.
  arXiv  Detail & Related papers  (2021-06-23T13:40:13Z)
• Encouraging Neural Machine Translation to Satisfy Terminology Constraints [3.3108924994485096]
  We present a new approach to encourage neural machine translation to satisfy lexical constraints. Our method acts at the training step and thereby avoiding the introduction of any extra computational overhead at inference step.
  arXiv  Detail & Related papers  (2021-06-07T15:46:07Z)
• On Sampling-Based Training Criteria for Neural Language Modeling [97.35284042981675]
  We consider Monte Carlo sampling, importance sampling, a novel method we call compensated partial summation, and noise contrastive estimation. We show that all these sampling methods can perform equally well, as long as we correct for the intended class posterior probabilities. Experimental results in language modeling and automatic speech recognition on Switchboard and LibriSpeech support our claim.
  arXiv  Detail & Related papers  (2021-04-21T12:55:52Z)
• Neural Text Generation with Artificial Negative Examples [7.187858820534111]
  We propose to suppress an arbitrary type of errors by training the text generation model in a reinforcement learning framework. We use a trainable reward function that is capable of discriminating between references and sentences containing the targeted type of errors. The experimental results show that our method can suppress the generation errors and achieve significant improvements on two machine translation and two image captioning tasks.
  arXiv  Detail & Related papers  (2020-12-28T07:25:10Z)
• Lexically Constrained Neural Machine Translation with Levenshtein Transformer [8.831954614241234]
  This paper proposes a simple and effective algorithm for incorporating lexical constraints in neural machine translation. Our method injects terminology constraints at inference time without any impact on decoding speed.
  arXiv  Detail & Related papers  (2020-04-27T09:59:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.