Cascaded Beam Search: Plug-and-Play Terminology-Forcing For Neural Machine Translation

• URL: http://arxiv.org/abs/2305.14538v1
• Date: Tue, 23 May 2023 21:48:02 GMT
• Title: Cascaded Beam Search: Plug-and-Play Terminology-Forcing For Neural Machine Translation
• Authors: Fr\'ed\'eric Odermatt and B\'eni Egressy and Roger Wattenhofer
• Abstract summary: This paper presents a plug-and-play approach for translation with terminology constraints. We propose Cascade Beam Search, a terminology-forcing approach that requires no training. We evaluate the performance of our approach by competing against the top submissions of the WMT21 terminology translation task.
• Score: 11.902884131696783
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a plug-and-play approach for translation with terminology constraints. Terminology constraints are an important aspect of many modern translation pipelines. In both specialized domains and newly emerging domains (such as the COVID-19 pandemic), accurate translation of technical terms is crucial. Recent approaches often train models to copy terminologies from the input into the output sentence by feeding the target terminology along with the input. But this requires expensive training whenever the underlying language model is changed or the system should specialize to a new domain. We propose Cascade Beam Search, a plug-and-play terminology-forcing approach that requires no training. Cascade Beam Search has two parts: 1) logit manipulation to increase the probability of target terminologies and 2) a cascading beam setup based on grid beam search, where beams are grouped by the number of terminologies they contain. We evaluate the performance of our approach by competing against the top submissions of the WMT21 terminology translation task. Our plug-and-play approach performs on par with the winning submissions without using a domain-specific language model and with no additional training.

Related papers

• Efficient Terminology Integration for LLM-based Translation in Specialized Domains [0.0]
  In specialized fields such as patent, finance, or biomedical domains, terminology is crucial for translation. We introduce a methodology that efficiently trains models with a smaller amount of data while preserving the accuracy of terminology translation. This methodology enhances the model's ability to handle specialized terminology and ensures high-quality translations.
  arXiv  Detail & Related papers  (2024-10-21T07:01:25Z)
• Plug, Play, and Fuse: Zero-Shot Joint Decoding via Word-Level Re-ranking Across Diverse Vocabularies [12.843274390224853]
  Real-world tasks, like multimodal translation, often require a combination of these strengths, such as handling both translation and image processing. We propose a novel zero-shot ensembling strategy that allows for the integration of different models during the decoding phase without the need for additional training. Our approach re-ranks beams during decoding by combining scores at the word level, using multimodals to predict when a word is completed.
  arXiv  Detail & Related papers  (2024-08-21T04:20:55Z)
• Terminology-Aware Translation with Constrained Decoding and Large Language Model Prompting [11.264272119913311]
  We submit to the WMT 2023 terminology translation task. We adopt a translate-then-refine approach which can be domain-independent and requires minimal manual efforts. Results show that our terminology-aware model learns to incorporate terminologies effectively.
  arXiv  Detail & Related papers  (2023-10-09T16:08:23Z)
• Towards Effective Disambiguation for Machine Translation with Large Language Models [65.80775710657672]
  We study the capabilities of large language models to translate "ambiguous sentences" Experiments show that our methods can match or outperform state-of-the-art systems such as DeepL and NLLB in four out of five language directions.
  arXiv  Detail & Related papers  (2023-09-20T22:22:52Z)
• DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition [94.90258603217008]
  The MultiCoNER RNum2 shared task aims to tackle multilingual named entity recognition (NER) in fine-grained and noisy scenarios. Previous top systems in the MultiCoNER RNum1 either incorporate the knowledge bases or gazetteers. We propose a unified retrieval-augmented system (U-RaNER) for fine-grained multilingual NER.
  arXiv  Detail & Related papers  (2023-05-05T16:59:26Z)
• Fine-grained Semantic Alignment Network for Weakly Supervised Temporal Language Grounding [148.46348699343991]
  Temporal language grounding aims to localize a video segment in an untrimmed video based on a natural language description. Most of the existing weakly supervised methods generate a candidate segment set and learn cross-modal alignment through a MIL-based framework. We propose a novel candidate-free framework: Fine-grained Semantic Alignment Network (FSAN), for weakly supervised TLG.
  arXiv  Detail & Related papers  (2022-10-21T13:10:27Z)
• DICTDIS: Dictionary Constrained Disambiguation for Improved NMT [50.888881348723295]
  We present DictDis, a lexically constrained NMT system that disambiguates between multiple candidate translations derived from dictionaries. We demonstrate the utility of DictDis via extensive experiments on English-Hindi and English-German sentences in a variety of domains including regulatory, finance, engineering.
  arXiv  Detail & Related papers  (2022-10-13T13:04:16Z)
• Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategies [72.56158036639707]
  Morphologically rich languages pose difficulties to machine translation. A large amount of differently inflected word surface forms entails a larger vocabulary. Some inflected forms of infrequent terms typically do not appear in the training corpus. Linguistic agreement requires the system to correctly match the grammatical categories between inflected word forms in the output sentence.
  arXiv  Detail & Related papers  (2022-03-25T10:13:20Z)
• CUNI systems for WMT21: Terminology translation Shared Task [0.0]
  The objective of this task is to design a system which translates certain terms based on a provided terminology database. Our approach is based on providing the desired translations alongside the input sentence and training the model to use these provided terms. We lemmatize the terms both during the training and inference, to allow the model to learn how to produce correct surface forms of the words.
  arXiv  Detail & Related papers  (2021-09-20T08:05:39Z)
• Improving Lexically Constrained Neural Machine Translation with Source-Conditioned Masked Span Prediction [6.46964825569749]
  In this paper, we tackle a more challenging setup consisting of domain-specific corpora with much longer n-gram and highly specialized terms. To encourage span-level representations in generation, we additionally impose a source-sentence conditioned masked span prediction loss in the decoder. Experimental results on three domain-specific corpora in two language pairs demonstrate that the proposed training scheme can improve the performance of existing lexically constrained methods.
  arXiv  Detail & Related papers  (2021-05-12T08:11:33Z)
• Language Guided Networks for Cross-modal Moment Retrieval [66.49445903955777]
  Cross-modal moment retrieval aims to localize a temporal segment from an untrimmed video described by a natural language query. Existing methods independently extract the features of videos and sentences. We present Language Guided Networks (LGN), a new framework that leverages the sentence embedding to guide the whole process of moment retrieval.
  arXiv  Detail & Related papers  (2020-06-18T12:08:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.