Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation

• URL: http://arxiv.org/abs/2511.21517v1
• Date: Wed, 26 Nov 2025 15:48:04 GMT
• Title: Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation
• Authors: Lina Conti, Dennis Fucci, Marco Gaido, Matteo Negri, Guillaume Wisniewski, Luisa Bentivogli,
• Abstract summary: We investigate the mechanisms ST models use to assign gender to speaker-referring terms across three language pairs.<n>We find that models do not simply replicate term-specific gender associations from training data, but learn broader patterns of masculine prevalence.<n>Using contrastive feature attribution on spectrograms, we reveal that the model with higher gender accuracy relies on a previously unknown mechanism.
• Score: 25.126933196101703
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unlike text, speech conveys information about the speaker, such as gender, through acoustic cues like pitch. This gives rise to modality-specific bias concerns. For example, in speech translation (ST), when translating from languages with notional gender, such as English, into languages where gender-ambiguous terms referring to the speaker are assigned grammatical gender, the speaker's vocal characteristics may play a role in gender assignment. This risks misgendering speakers, whether through masculine defaults or vocal-based assumptions. Yet, how ST models make these decisions remains poorly understood. We investigate the mechanisms ST models use to assign gender to speaker-referring terms across three language pairs (en-es/fr/it), examining how training data patterns, internal language model (ILM) biases, and acoustic information interact. We find that models do not simply replicate term-specific gender associations from training data, but learn broader patterns of masculine prevalence. While the ILM exhibits strong masculine bias, models can override these preferences based on acoustic input. Using contrastive feature attribution on spectrograms, we reveal that the model with higher gender accuracy relies on a previously unknown mechanism: using first-person pronouns to link gendered terms back to the speaker, accessing gender information distributed across the frequency spectrum rather than concentrated in pitch.

Related papers

• Acoustic-based Gender Differentiation in Speech-aware Language Models [3.9845890275228277]
  Speech-aware Language Models (SpeechLMs) have fundamentally transformed human-AI interaction by enabling voice-based communication.<n>This paper propose a new dataset that enables systematic analysis of this phenomenon, containing 9,208 speech samples across three categories: Gender-Independent, Gender-Stereotypical, and Gender-Dependent.
  arXiv  Detail & Related papers  (2025-09-25T13:15:01Z)
• Who Gets the Mic? Investigating Gender Bias in the Speaker Assignment of a Speech-LLM [4.12691471378072]
  This study proposes a methodology leveraging speaker assignment as an analytic tool for bias investigation.<n>We evaluate Bark, a Text-to-Speech (TTS) model, analyzing its default speaker assignments for textual prompts.<n>If Bark's speaker selection systematically aligns with gendered associations, it may reveal patterns in its training data or model design.
  arXiv  Detail & Related papers  (2025-08-19T08:10:55Z)
• Gender Bias in Instruction-Guided Speech Synthesis Models [55.2480439325792]
  This study investigates the potential gender bias in how models interpret occupation-related prompts.<n>We explore whether these models exhibit tendencies to amplify gender stereotypes when interpreting such prompts.<n>Our experimental results reveal the model's tendency to exhibit gender bias for certain occupations.
  arXiv  Detail & Related papers  (2025-02-08T17:38:24Z)
• Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models [50.40276881893513]
  This study introduces Spoken Stereoset, a dataset specifically designed to evaluate social biases in Speech Large Language Models (SLLMs) By examining how different models respond to speech from diverse demographic groups, we aim to identify these biases. The findings indicate that while most models show minimal bias, some still exhibit slightly stereotypical or anti-stereotypical tendencies.
  arXiv  Detail & Related papers  (2024-08-14T16:55:06Z)
• Beyond Binary Gender: Evaluating Gender-Inclusive Machine Translation with Ambiguous Attitude Words [85.48043537327258]
  Existing machine translation gender bias evaluations are primarily focused on male and female genders. This study presents a benchmark AmbGIMT (Gender-Inclusive Machine Translation with Ambiguous attitude words) We propose a novel process to evaluate gender bias based on the Emotional Attitude Score (EAS), which is used to quantify ambiguous attitude words.
  arXiv  Detail & Related papers  (2024-07-23T08:13:51Z)
• Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection [23.993869026482415]
  We propose the first inference-time solution to control speaker-related gender inflections in speech translation. Our solution partially replaces the (biased) internal language model (LM) implicitly learned by the ST decoder with gender-specific external LMs.
  arXiv  Detail & Related papers  (2023-10-24T11:55:16Z)
• How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation [21.125217707038356]
  When translating from notional gender languages into grammatical gender languages, the generated translation requires explicit gender assignments for various words, including those referring to the speaker. To avoid such biased and not inclusive behaviors, the gender assignment of speaker-related expressions should be guided by externally-provided metadata about the speaker's gender. This paper aims to achieve the same results by integrating the speaker's gender metadata into a single "multi-gender" neural ST model, easier to maintain.
  arXiv  Detail & Related papers  (2023-10-23T17:21:32Z)
• Analyzing Gender Representation in Multilingual Models [59.21915055702203]
  We focus on the representation of gender distinctions as a practical case study. We examine the extent to which the gender concept is encoded in shared subspaces across different languages.
  arXiv  Detail & Related papers  (2022-04-20T00:13:01Z)
• Protecting gender and identity with disentangled speech representations [49.00162808063399]
  We show that protecting gender information in speech is more effective than modelling speaker-identity information. We present a novel way to encode gender information and disentangle two sensitive biometric identifiers.
  arXiv  Detail & Related papers  (2021-04-22T13:31:41Z)
• Breeding Gender-aware Direct Speech Translation Systems [14.955696163410254]
  We show that gender-aware direct ST solutions can significantly outperform strong - but gender-unaware - direct ST models. The translation of gender-marked words can increase up to 30 points in accuracy while preserving overall translation quality.
  arXiv  Detail & Related papers  (2020-12-09T10:18:03Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.