It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information

• URL: http://arxiv.org/abs/2005.02354v2
• Date: Sun, 17 May 2020 06:59:57 GMT
• Title: It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information
• Authors: Emanuele Bugliarello, Sabrina J. Mielke, Antonios Anastasopoulos, Ryan Cotterell, Naoaki Okazaki
• Abstract summary: Cross-mutual information (XMI) is an asymmetric information-theoretic metric of machine translation difficulty. XMI exploits the probabilistic nature of most neural machine translation models. We present the first systematic and controlled study of cross-lingual translation difficulties using modern neural translation systems.
• Score: 90.35685796083563
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The performance of neural machine translation systems is commonly evaluated in terms of BLEU. However, due to its reliance on target language properties and generation, the BLEU metric does not allow an assessment of which translation directions are more difficult to model. In this paper, we propose cross-mutual information (XMI): an asymmetric information-theoretic metric of machine translation difficulty that exploits the probabilistic nature of most neural machine translation models. XMI allows us to better evaluate the difficulty of translating text into the target language while controlling for the difficulty of the target-side generation component independent of the translation task. We then present the first systematic and controlled study of cross-lingual translation difficulties using modern neural translation systems. Code for replicating our experiments is available online at https://github.com/e-bug/nmt-difficulty.

Related papers

• LANDeRMT: Detecting and Routing Language-Aware Neurons for Selectively Finetuning LLMs to Machine Translation [43.26446958873554]
  Large language models (LLMs) have shown promising results in multilingual translation even with limited bilingual supervision. Recent advancements in large language models (LLMs) have shown promising results in multilingual translation even with limited bilingual supervision. LandeRMT is a framework that selectively finetunes LLMs to textbfMachine textbfTranslation with diverse translation training data.
  arXiv  Detail & Related papers  (2024-09-29T02:39:42Z)
• The Inside Story: Towards Better Understanding of Machine Translation Neural Evaluation Metrics [8.432864879027724]
  We develop and compare several neural explainability methods and demonstrate their effectiveness for interpreting state-of-the-art fine-tuned neural metrics. Our study reveals that these metrics leverage token-level information that can be directly attributed to translation errors.
  arXiv  Detail & Related papers  (2023-05-19T16:42:17Z)
• Contrastive Learning for Context-aware Neural Machine TranslationUsing Coreference Information [14.671424999873812]
  We propose CorefCL, a novel data augmentation and contrastive learning scheme based on coreference between the source and contextual sentences. By corrupting automatically detected coreference mentions in the contextual sentence, CorefCL can train the model to be sensitive to coreference inconsistency. In experiments, our method consistently improved BLEU of compared models on English-German and English-Korean tasks.
  arXiv  Detail & Related papers  (2021-09-13T05:18:47Z)
• Improving Multilingual Translation by Representation and Gradient Regularization [82.42760103045083]
  We propose a joint approach to regularize NMT models at both representation-level and gradient-level. Our results demonstrate that our approach is highly effective in both reducing off-target translation occurrences and improving zero-shot translation performance.
  arXiv  Detail & Related papers  (2021-09-10T10:52:21Z)
• ChrEnTranslate: Cherokee-English Machine Translation Demo with Quality Estimation and Corrective Feedback [70.5469946314539]
  ChrEnTranslate is an online machine translation demonstration system for translation between English and an endangered language Cherokee. It supports both statistical and neural translation models as well as provides quality estimation to inform users of reliability.
  arXiv  Detail & Related papers  (2021-07-30T17:58:54Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• Exploiting Neural Query Translation into Cross Lingual Information Retrieval [49.167049709403166]
  Existing CLIR systems mainly exploit statistical-based machine translation (SMT) rather than the advanced neural machine translation (NMT) We propose a novel data augmentation method that extracts query translation pairs according to user clickthrough data. Experimental results reveal that the proposed approach yields better retrieval quality than strong baselines.
  arXiv  Detail & Related papers  (2020-10-26T15:28:19Z)
• Towards Multimodal Simultaneous Neural Machine Translation [28.536262015508722]
  Simultaneous translation involves translating a sentence before the speaker's utterance is completed in order to realize real-time understanding. This task is significantly more challenging than the general full sentence translation because of the shortage of input information during decoding. We propose multimodal simultaneous neural machine translation (MSNMT), which leverages visual information as an additional modality.
  arXiv  Detail & Related papers  (2020-04-07T08:02:21Z)
• Cross-lingual Supervision Improves Unsupervised Neural Machine Translation [97.84871088440102]
  We introduce a multilingual unsupervised NMT framework to leverage weakly supervised signals from high-resource language pairs to zero-resource translation directions. Method significantly improves the translation quality by more than 3 BLEU score on six benchmark unsupervised translation directions.
  arXiv  Detail & Related papers  (2020-04-07T05:46:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.