Repairing Pronouns in Translation with BERT-Based Post-Editing

• URL: http://arxiv.org/abs/2103.12838v2
• Date: Thu, 25 Mar 2021 13:21:09 GMT
• Title: Repairing Pronouns in Translation with BERT-Based Post-Editing
• Authors: Reid Pryzant, Melvin Johnson, Hideto Kazawa
• Abstract summary: We show that in some domains, pronoun choice can account for more than half of a NMT systems' errors. We then investigate a possible solution: fine-tuning BERT on a pronoun prediction task using chunks of source-side sentences.
• Score: 7.6344611819427035
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pronouns are important determinants of a text's meaning but difficult to translate. This is because pronoun choice can depend on entities described in previous sentences, and in some languages pronouns may be dropped when the referent is inferrable from the context. These issues can lead Neural Machine Translation (NMT) systems to make critical errors on pronouns that impair intelligibility and even reinforce gender bias. We investigate the severity of this pronoun issue, showing that (1) in some domains, pronoun choice can account for more than half of a NMT systems' errors, and (2) pronouns have a disproportionately large impact on perceived translation quality. We then investigate a possible solution: fine-tuning BERT on a pronoun prediction task using chunks of source-side sentences, then using the resulting classifier to repair the translations of an existing NMT model. We offer an initial case study of this approach for the Japanese-English language pair, observing that a small number of translations are significantly improved according to human evaluators.

Related papers

• Understanding and Addressing the Under-Translation Problem from the Perspective of Decoding Objective [72.83966378613238]
  Under-translation and over-translation remain two challenging problems in state-of-the-art Neural Machine Translation (NMT) systems. We conduct an in-depth analysis on the underlying cause of under-translation in NMT, providing an explanation from the perspective of decoding objective. We propose employing the confidence of predicting End Of Sentence (EOS) as a detector for under-translation, and strengthening the confidence-based penalty to penalize candidates with a high risk of under-translation.
  arXiv  Detail & Related papers  (2024-05-29T09:25:49Z)
• A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation [35.44115368160656]
  We investigate whether and to what extent machine translation models exhibit gender bias. We find that IFT models default to male-inflected translations, even disregarding female occupational stereotypes. We propose an easy-to-implement and effective bias mitigation solution.
  arXiv  Detail & Related papers  (2023-10-18T17:36:55Z)
• Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting [66.02718577386426]
  We provide a simple characterization of idiomatic translation and related issues. We conduct a synthetic experiment revealing a tipping point at which transformer-based machine translation models correctly default to idiomatic translations. To improve translation of natural idioms, we introduce two straightforward yet effective techniques.
  arXiv  Detail & Related papers  (2023-10-10T23:47:25Z)
• Extract and Attend: Improving Entity Translation in Neural Machine Translation [141.7840980565706]
  We propose an Extract-and-Attend approach to enhance entity translation in NMT. The proposed method is effective on improving both the translation accuracy of entities and the overall translation quality.
  arXiv  Detail & Related papers  (2023-06-04T03:05:25Z)
• A Survey on Zero Pronoun Translation [69.09774294082965]
  Zero pronouns (ZPs) are frequently omitted in pro-drop languages, but should be recalled in non-pro-drop languages. This survey paper highlights the major works that have been undertaken in zero pronoun translation (ZPT) after the neural revolution. We uncover a number of insightful findings such as: 1) ZPT is in line with the development trend of large language model; 2) data limitation causes learning bias in languages and domains; 3) performance improvements are often reported on single benchmarks, but advanced methods are still far from real-world use.
  arXiv  Detail & Related papers  (2023-05-17T13:19:01Z)
• Rethink about the Word-level Quality Estimation for Machine Translation from Human Judgement [57.72846454929923]
  We create a benchmark dataset, emphHJQE, where the expert translators directly annotate poorly translated words. We propose two tag correcting strategies, namely tag refinement strategy and tree-based annotation strategy, to make the TER-based artificial QE corpus closer to emphHJQE. The results show our proposed dataset is more consistent with human judgement and also confirm the effectiveness of the proposed tag correcting strategies.
  arXiv  Detail & Related papers  (2022-09-13T02:37:12Z)
• How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts [11.684346035745975]
  A growing number of studies highlight the inherent gender bias that Neural Machine Translation models incorporate during training. We investigate whether these models can be instructed to fix their bias during inference using targeted, guided instructions as contexts. We observe large improvements in reducing the gender bias in translations, across three popular test suites.
  arXiv  Detail & Related papers  (2022-05-22T06:31:54Z)
• Mitigating Gender Bias in Machine Translation through Adversarial Learning [0.8883733362171032]
  We present an adversarial learning framework that addresses challenges to mitigate gender bias in seq2seq machine translation. Our framework improves the disparity in translation quality for sentences with male vs. female entities by 86% for English-German translation and 91% for English-French translation.
  arXiv  Detail & Related papers  (2022-03-20T23:35:09Z)
• DEEP: DEnoising Entity Pre-training for Neural Machine Translation [123.6686940355937]
  It has been shown that machine translation models usually generate poor translations for named entities that are infrequent in the training corpus. We propose DEEP, a DEnoising Entity Pre-training method that leverages large amounts of monolingual data and a knowledge base to improve named entity translation accuracy within sentences.
  arXiv  Detail & Related papers  (2021-11-14T17:28:09Z)
• Investigating Failures of Automatic Translation in the Case of Unambiguous Gender [13.58884863186619]
  Transformer based models are the modern work horses for neural machine translation (NMT) We observe a systemic and rudimentary class of errors made by transformer based models with regards to translating from a language that doesn't mark gender on nouns into others that do. We release an evaluation scheme and dataset for measuring the ability of transformer based NMT models to translate gender correctly.
  arXiv  Detail & Related papers  (2021-04-16T00:57:36Z)
• Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation [4.775445987662862]
  Machine translation systems with inadequate document understanding can make errors when translating dropped or neutral pronouns into languages with gendered pronouns. We propose a novel cross-lingual pivoting technique for automatically producing high-quality gender labels.
  arXiv  Detail & Related papers  (2020-06-16T02:41:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.