Significance of Chain of Thought in Gender Bias Mitigation for English-Dravidian Machine Translation

• URL: http://arxiv.org/abs/2405.19701v2
• Date: Mon, 3 Jun 2024 15:59:34 GMT
• Title: Significance of Chain of Thought in Gender Bias Mitigation for English-Dravidian Machine Translation
• Authors: Lavanya Prahallad, Radhika Mamidi,
• Abstract summary: This paper examines gender bias in machine translation systems for languages such as Telugu and Kan- nada from the Dravidian family. It finds that while plural forms can reduce bias, individual-centric sentences often main- tain the bias due to historical stereotypes.
• Score: 6.200058263544999
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gender bias in machine translation (MT) sys- tems poses a significant challenge to achieving accurate and inclusive translations. This paper examines gender bias in machine translation systems for languages such as Telugu and Kan- nada from the Dravidian family, analyzing how gender inflections affect translation accuracy and neutrality using Google Translate and Chat- GPT. It finds that while plural forms can reduce bias, individual-centric sentences often main- tain the bias due to historical stereotypes. The study evaluates the Chain of Thought process- ing, noting significant bias mitigation from 80% to 4% in Telugu and from 40% to 0% in Kan- nada. It also compares Telugu and Kannada translations, emphasizing the need for language specific strategies to address these challenges and suggesting directions for future research to enhance fairness in both data preparation and prompts during inference.

Related papers

• Beyond Binary Gender: Evaluating Gender-Inclusive Machine Translation with Ambiguous Attitude Words [85.48043537327258]
  Existing machine translation gender bias evaluations are primarily focused on male and female genders. This study presents a benchmark AmbGIMT (Gender-Inclusive Machine Translation with Ambiguous attitude words) We propose a novel process to evaluate gender bias based on the Emotional Attitude Score (EAS), which is used to quantify ambiguous attitude words.
  arXiv  Detail & Related papers  (2024-07-23T08:13:51Z)
• Investigating Markers and Drivers of Gender Bias in Machine Translations [0.0]
  Implicit gender bias in large language models (LLMs) is a well-documented problem. We use the DeepL translation API to investigate the bias evinced when repeatedly translating a set of 56 Software Engineering tasks. We find that some languages display similar patterns of pronoun use, falling into three loose groups. We identify the main verb appearing in a sentence as a likely significant driver of implied gender in the translations.
  arXiv  Detail & Related papers  (2024-03-18T15:54:46Z)
• GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered Languages [0.0]
  We introduce GATE X-E, an extension to the GATE corpus, that consists of human translations from Turkish, Hungarian, Finnish, and Persian into English. The dataset features natural sentences with a wide range of sentence lengths and domains, challenging translation rewriters on various linguistic phenomena. We present a translation gender rewriting solution built with GPT-4 and use GATE X-E to evaluate it.
  arXiv  Detail & Related papers  (2024-02-22T04:36:14Z)
• Gender Bias in Machine Translation and The Era of Large Language Models [0.8702432681310399]
  This chapter examines the role of Machine Translation in perpetuating gender bias, highlighting the challenges posed by cross-linguistic settings and statistical dependencies. A comprehensive overview of relevant existing work related to gender bias in both conventional Neural Machine Translation approaches and Generative Pretrained Transformer models employed as Machine Translation systems is provided.
  arXiv  Detail & Related papers  (2024-01-18T14:34:49Z)
• Evaluating Gender Bias in the Translation of Gender-Neutral Languages into English [0.0]
  We introduce GATE X-E, an extension to the GATE corpus, that consists of human translations from Turkish, Hungarian, Finnish, and Persian into English. The dataset features natural sentences with a wide range of sentence lengths and domains, challenging translation rewriters on various linguistic phenomena. We present an English gender rewriting solution built on GPT-3.5 Turbo and use GATE X-E to evaluate it.
  arXiv  Detail & Related papers  (2023-11-15T10:25:14Z)
• A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation [35.44115368160656]
  We investigate whether and to what extent machine translation models exhibit gender bias. We find that IFT models default to male-inflected translations, even disregarding female occupational stereotypes. We propose an easy-to-implement and effective bias mitigation solution.
  arXiv  Detail & Related papers  (2023-10-18T17:36:55Z)
• The Gender-GAP Pipeline: A Gender-Aware Polyglot Pipeline for Gender Characterisation in 55 Languages [51.2321117760104]
  This paper describes the Gender-GAP Pipeline, an automatic pipeline to characterize gender representation in large-scale datasets for 55 languages. The pipeline uses a multilingual lexicon of gendered person-nouns to quantify the gender representation in text. We showcase it to report gender representation in WMT training data and development data for the News task, confirming that current data is skewed towards masculine representation.
  arXiv  Detail & Related papers  (2023-08-31T17:20:50Z)
• Counter-GAP: Counterfactual Bias Evaluation through Gendered Ambiguous Pronouns [53.62845317039185]
  Bias-measuring datasets play a critical role in detecting biased behavior of language models. We propose a novel method to collect diverse, natural, and minimally distant text pairs via counterfactual generation. We show that four pre-trained language models are significantly more inconsistent across different gender groups than within each group.
  arXiv  Detail & Related papers  (2023-02-11T12:11:03Z)
• On the Language Coverage Bias for Neural Machine Translation [81.81456880770762]
  Language coverage bias is important for neural machine translation (NMT) because the target-original training data is not well exploited in current practice. By carefully designing experiments, we provide comprehensive analyses of the language coverage bias in the training data. We propose two simple and effective approaches to alleviate the language coverage bias problem.
  arXiv  Detail & Related papers  (2021-06-07T01:55:34Z)
• Decoding and Diversity in Machine Translation [90.33636694717954]
  We characterize differences between cost diversity paid for the BLEU scores enjoyed by NMT. Our study implicates search as a salient source of known bias when translating gender pronouns.
  arXiv  Detail & Related papers  (2020-11-26T21:09:38Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.