Gender-specific Machine Translation with Large Language Models

• URL: http://arxiv.org/abs/2309.03175v2
• Date: Tue, 16 Apr 2024 19:16:46 GMT
• Title: Gender-specific Machine Translation with Large Language Models
• Authors: Eduardo Sánchez, Pierre Andrews, Pontus Stenetorp, Mikel Artetxe, Marta R. Costa-jussà,
• Abstract summary: Decoder-only Large Language Models (LLMs) have demonstrated potential in machine translation. LLMs offer the ability to control the properties of the output through prompts. Our results indicate that LLaMa can generate gender-specific translations with translation accuracy and gender bias comparable to state-of-the-art multilingual NMT system.
• Score: 39.49764957694078
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: While machine translation (MT) systems have seen significant improvements, it is still common for translations to reflect societal biases, such as gender bias. Decoder-only Large Language Models (LLMs) have demonstrated potential in MT, albeit with performance slightly lagging behind traditional encoder-decoder Neural Machine Translation (NMT) systems. However, LLMs offer a unique advantage: the ability to control the properties of the output through prompts. In this study, we leverage this flexibility to explore LLaMa's capability to produce gender-specific translations. Our results indicate that LLaMa can generate gender-specific translations with translation accuracy and gender bias comparable to NLLB, a state-of-the-art multilingual NMT system. Furthermore, our experiments reveal that LLaMa's gender-specific translations rely on coreference resolution to determine gender, showing higher gender variance in gender-ambiguous datasets but maintaining consistency in less ambiguous contexts. This research investigates the potential and challenges of using LLMs for gender-specific translations as an instance of the controllability of outputs offered by LLMs.

Related papers

• The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs [1.707677607445317]
  This paper studies gender bias in machine translation through the lens of Large Language Models (LLMs) Four widely-used test sets are employed to benchmark various base LLMs, comparing their translation quality and gender bias against state-of-the-art Neural Machine Translation (NMT) models for English to Catalan (En $rightarrow$ Ca) and English to Spanish (En $rightarrow$ Es) Our findings reveal pervasive gender bias across all models, with base LLMs exhibiting a higher degree of bias compared to NMT models.
  arXiv  Detail & Related papers  (2024-07-26T14:47:31Z)
• Fine-grained Gender Control in Machine Translation with Large Language Models [15.63784352130237]
  We tackle controlled translation in a more realistic setting of inputs with multiple entities. Our proposed method instructs the model with fine-grained entity-level gender information to translate with correct gender inflections. We discover an emergence of gender interference phenomenon when controlling the gender of multiple entities.
  arXiv  Detail & Related papers  (2024-07-21T13:15:00Z)
• Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding [73.32763904267186]
  Large Language Models (LLMs) present the potential for achieving superior translation quality. We propose Cooperative Decoding (CoDec) which treats NMT systems as a pretranslation model and MT-oriented LLMs as a supplemental solution.
  arXiv  Detail & Related papers  (2023-11-06T03:41:57Z)
• Probing Explicit and Implicit Gender Bias through LLM Conditional Text Generation [64.79319733514266]
  Large Language Models (LLMs) can generate biased and toxic responses. We propose a conditional text generation mechanism without the need for predefined gender phrases and stereotypes.
  arXiv  Detail & Related papers  (2023-11-01T05:31:46Z)
• A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation [35.44115368160656]
  We investigate whether and to what extent machine translation models exhibit gender bias. We find that IFT models default to male-inflected translations, even disregarding female occupational stereotypes. We propose an easy-to-implement and effective bias mitigation solution.
  arXiv  Detail & Related papers  (2023-10-18T17:36:55Z)
• Towards Effective Disambiguation for Machine Translation with Large Language Models [65.80775710657672]
  We study the capabilities of large language models to translate "ambiguous sentences" Experiments show that our methods can match or outperform state-of-the-art systems such as DeepL and NLLB in four out of five language directions.
  arXiv  Detail & Related papers  (2023-09-20T22:22:52Z)
• Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis [103.89753784762445]
  Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT) This paper systematically investigates the advantages and challenges of LLMs for MMT. We thoroughly evaluate eight popular LLMs, including ChatGPT and GPT-4.
  arXiv  Detail & Related papers  (2023-04-10T15:51:30Z)
• Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation [91.57514888410205]
  Large language models (LLMs) demonstrate remarkable machine translation (MT) abilities via prompting. LLMs can struggle to translate inputs with rare words, which are common in low resource or domain transfer scenarios. We show that LLM prompting can provide an effective solution for rare words as well, by using prior knowledge from bilingual dictionaries to provide control hints in the prompts.
  arXiv  Detail & Related papers  (2023-02-15T18:46:42Z)
• Investigating Failures of Automatic Translation in the Case of Unambiguous Gender [13.58884863186619]
  Transformer based models are the modern work horses for neural machine translation (NMT) We observe a systemic and rudimentary class of errors made by transformer based models with regards to translating from a language that doesn't mark gender on nouns into others that do. We release an evaluation scheme and dataset for measuring the ability of transformer based NMT models to translate gender correctly.
  arXiv  Detail & Related papers  (2021-04-16T00:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.