The Voice Behind the Words: Quantifying Intersectional Bias in SpeechLLMs

• URL: http://arxiv.org/abs/2603.16941v1
• Date: Sun, 15 Mar 2026 16:54:36 GMT
• Title: The Voice Behind the Words: Quantifying Intersectional Bias in SpeechLLMs
• Abstract summary: Speech Large Language Models (SpeechLLMs) process spoken input directly, retaining cues such as accent and perceived gender.<n>We present a large-scale intersectional evaluation of accent and gender bias in three SpeechLLMs using 2,880 controlled interactions.
• Score: 34.359037939013085
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speech Large Language Models (SpeechLLMs) process spoken input directly, retaining cues such as accent and perceived gender that were previously removed in cascaded pipelines. This introduces speaker identity dependent variation in responses. We present a large-scale intersectional evaluation of accent and gender bias in three SpeechLLMs using 2,880 controlled interactions across six English accents and two gender presentations, keeping linguistic content constant through voice cloning. Using pointwise LLM-judge ratings, pairwise comparisons, and Best-Worst Scaling with human validation, we detect consistent disparities. Eastern European-accented speech receives lower helpfulness scores, particularly for female-presenting voices. The bias is implicit: responses remain polite but differ in helpfulness. While LLM judges capture the directional trend of these biases, human evaluators exhibit significantly higher sensitivity, uncovering sharper intersectional disparities.

Related papers

• From Seeing it to Experiencing it: Interactive Evaluation of Intersectional Voice Bias in Human-AI Speech Interaction [25.529747060487363]
  SpeechLLMs process spoken language directly from audio, but accent and vocal identity cues can lead to biased behaviour.<n>We distinguish quality-of-service disparities from content-level bias in coherent outputs and examine intersectional effects of accent and perceived gender.
  arXiv  Detail & Related papers  (2026-03-19T21:12:49Z)
• Speak Your Mind: The Speech Continuation Task as a Probe of Voice-Based Model Bias [24.932603485660323]
  Speech Continuation (SC) is the task of generating a coherent extension of a spoken prompt while preserving semantic context and speaker identity.<n>We present the first systematic evaluation of bias in SC, investigating how gender and phonation type affect continuation behaviour.
  arXiv  Detail & Related papers  (2025-09-26T08:43:25Z)
• Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models [50.40276881893513]
  This study introduces Spoken Stereoset, a dataset specifically designed to evaluate social biases in Speech Large Language Models (SLLMs) By examining how different models respond to speech from diverse demographic groups, we aim to identify these biases. The findings indicate that while most models show minimal bias, some still exhibit slightly stereotypical or anti-stereotypical tendencies.
  arXiv  Detail & Related papers  (2024-08-14T16:55:06Z)
• Beyond Binary Gender: Evaluating Gender-Inclusive Machine Translation with Ambiguous Attitude Words [85.48043537327258]
  Existing machine translation gender bias evaluations are primarily focused on male and female genders. This study presents a benchmark AmbGIMT (Gender-Inclusive Machine Translation with Ambiguous attitude words) We propose a novel process to evaluate gender bias based on the Emotional Attitude Score (EAS), which is used to quantify ambiguous attitude words.
  arXiv  Detail & Related papers  (2024-07-23T08:13:51Z)
• Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gaps [25.95711246919163]
  Current automatic speech recognition (ASR) models are designed to be used across many languages and tasks without substantial changes. Our study systematically evaluates the performance of two widely used multilingual ASR models on three datasets. Our findings reveal clear gender disparities, with the advantaged group varying across languages and models.
  arXiv  Detail & Related papers  (2024-02-28T00:24:29Z)
• No Pitch Left Behind: Addressing Gender Unbalance in Automatic Speech Recognition through Pitch Manipulation [20.731375136671605]
  We propose a data augmentation technique that manipulates the fundamental frequency (f0) and formants. This technique reduces the data unbalance among genders by simulating voices of the under-represented female speakers. Experiments on spontaneous English speech show that our technique yields a relative WER improvement up to 9.87% for utterances by female speakers.
  arXiv  Detail & Related papers  (2023-10-10T12:55:22Z)
• Generating Multilingual Gender-Ambiguous Text-to-Speech Voices [4.005334718121374]
  This work addresses the task of generating novel gender-ambiguous TTS voices in a multi-speaker, multilingual setting. To our knowledge, this is the first systematic and validated approach that can reliably generate a variety of gender-ambiguous voices.
  arXiv  Detail & Related papers  (2022-11-01T10:40:24Z)
• UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data [54.733889961024445]
  We propose a unified pre-training approach called UniSpeech to learn speech representations with both unlabeled and labeled data. We evaluate the effectiveness of UniSpeech for cross-lingual representation learning on public CommonVoice corpus.
  arXiv  Detail & Related papers  (2021-01-19T12:53:43Z)
• Unsupervised Cross-lingual Representation Learning for Speech Recognition [63.85924123692923]
  XLSR learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec 2.0 which is trained by solving a contrastive task over masked latent speech representations. Experiments show that cross-lingual pretraining significantly outperforms monolingual pretraining.
  arXiv  Detail & Related papers  (2020-06-24T18:25:05Z)
• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer [101.58431011820755]
  We study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications. We create a multilingual dataset for bias analysis and propose several ways for quantifying bias in multilingual representations.
  arXiv  Detail & Related papers  (2020-05-02T04:34:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.