Reducing Gender Bias in Machine Translation through Counterfactual Data Generation

• URL: http://arxiv.org/abs/2311.16362v1
• Date: Mon, 27 Nov 2023 23:03:01 GMT
• Title: Reducing Gender Bias in Machine Translation through Counterfactual Data Generation
• Authors: Ranjita Naik, Spencer Rarrick, Vishal Chowdhary
• Abstract summary: We show that gender bias can be significantly mitigated, albeit at the expense of translation quality due to catastrophic forgetting. We also propose a novel domain-adaptation technique that leverages in-domain data created with the counterfactual data generation techniques. The relevant code will be available at Github.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in neural methods have led to substantial improvement in the quality of Neural Machine Translation (NMT) systems. However, these systems frequently produce translations with inaccurate gender (Stanovsky et al., 2019), which can be traced to bias in training data. Saunders and Byrne (2020) tackle this problem with a handcrafted dataset containing balanced gendered profession words. By using this data to fine-tune an existing NMT model, they show that gender bias can be significantly mitigated, albeit at the expense of translation quality due to catastrophic forgetting. They recover some of the lost quality with modified training objectives or additional models at inference. We find, however, that simply supplementing the handcrafted dataset with a random sample from the base model training corpus is enough to significantly reduce the catastrophic forgetting. We also propose a novel domain-adaptation technique that leverages in-domain data created with the counterfactual data generation techniques proposed by Zmigrod et al. (2019) to further improve accuracy on the WinoMT challenge test set without significant loss in translation quality. We show its effectiveness in NMT systems from English into three morphologically rich languages French, Spanish, and Italian. The relevant dataset and code will be available at Github.

Related papers

• Towards Fine-Grained Information: Identifying the Type and Location of Translation Errors [80.22825549235556]
  Existing approaches can not synchronously consider error position and type. We build an FG-TED model to predict the textbf addition and textbfomission errors. Experiments show that our model can identify both error type and position concurrently, and gives state-of-the-art results.
  arXiv  Detail & Related papers  (2023-02-17T16:20:33Z)
• Better Datastore, Better Translation: Generating Datastores from Pre-Trained Models for Nearest Neural Machine Translation [48.58899349349702]
  Nearest Neighbor Machine Translation (kNNMT) is a simple and effective method of augmenting neural machine translation (NMT) with a token-level nearest neighbor retrieval mechanism. In this paper, we propose PRED, a framework that leverages Pre-trained models for Datastores in kNN-MT.
  arXiv  Detail & Related papers  (2022-12-17T08:34:20Z)
• How sensitive are translation systems to extra contexts? Mitigating gender bias in Neural Machine Translation models through relevant contexts [11.684346035745975]
  A growing number of studies highlight the inherent gender bias that Neural Machine Translation models incorporate during training. We investigate whether these models can be instructed to fix their bias during inference using targeted, guided instructions as contexts. We observe large improvements in reducing the gender bias in translations, across three popular test suites.
  arXiv  Detail & Related papers  (2022-05-22T06:31:54Z)
• Mitigating Gender Bias in Machine Translation through Adversarial Learning [0.8883733362171032]
  We present an adversarial learning framework that addresses challenges to mitigate gender bias in seq2seq machine translation. Our framework improves the disparity in translation quality for sentences with male vs. female entities by 86% for English-German translation and 91% for English-French translation.
  arXiv  Detail & Related papers  (2022-03-20T23:35:09Z)
• Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting [88.83117372793737]
  Forgetting information in the original training data may damage the model's downstream performance by a large margin. We propose GEnder Equality Prompt (GEEP) to improve gender fairness of pre-trained models with less forgetting.
  arXiv  Detail & Related papers  (2021-10-11T15:52:16Z)
• Exploring Unsupervised Pretraining Objectives for Machine Translation [99.5441395624651]
  Unsupervised cross-lingual pretraining has achieved strong results in neural machine translation (NMT) Most approaches adapt masked-language modeling (MLM) to sequence-to-sequence architectures, by masking parts of the input and reconstructing them in the decoder. We compare masking with alternative objectives that produce inputs resembling real (full) sentences, by reordering and replacing words based on their context.
  arXiv  Detail & Related papers  (2021-06-10T10:18:23Z)
• Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation [98.11249019844281]
  Knowledge distillation (KD) is commonly used to construct synthetic data for training non-autoregressive translation (NAT) models. We propose reverse KD to rejuvenate more alignments for low-frequency target words. Results demonstrate that the proposed approach can significantly and universally improve translation quality.
  arXiv  Detail & Related papers  (2021-06-02T02:41:40Z)
• Investigating Failures of Automatic Translation in the Case of Unambiguous Gender [13.58884863186619]
  Transformer based models are the modern work horses for neural machine translation (NMT) We observe a systemic and rudimentary class of errors made by transformer based models with regards to translating from a language that doesn't mark gender on nouns into others that do. We release an evaluation scheme and dataset for measuring the ability of transformer based NMT models to translate gender correctly.
  arXiv  Detail & Related papers  (2021-04-16T00:57:36Z)
• Can Automatic Post-Editing Improve NMT? [9.233407096706744]
  Automatic post-editing (APE) aims to improve machine translations, thereby reducing human post-editing effort. APE has had notable success when used with statistical machine translation (SMT) systems but has not been as successful over neural machine translation (NMT) systems.
  arXiv  Detail & Related papers  (2020-09-30T02:34:19Z)
• On the Inference Calibration of Neural Machine Translation [54.48932804996506]
  We study the correlation between calibration and translation performance and linguistic properties of miscalibration. We propose a new graduated label smoothing method that can improve both inference calibration and translation performance.
  arXiv  Detail & Related papers  (2020-05-03T02:03:56Z)
• Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem [21.44025591721678]
  Training data for NLP tasks often exhibits gender bias in that fewer sentences refer to women than to men. Recent WinoMT challenge set allows us to measure this effect directly. We use transfer learning on a small set of trusted, gender-balanced examples.
  arXiv  Detail & Related papers  (2020-04-09T11:55:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.