VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech

• URL: http://arxiv.org/abs/2604.17248v1
• Date: Sun, 19 Apr 2026 04:22:47 GMT
• Title: VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech
• Abstract summary: Large Audio-Language Models (LALMs) are increasingly integrated into daily applications, yet their generative biases remain underexplored.<n>We propose a framework that evaluates generative bias through open-ended tasks such as personalized recommendations, using real-world human recordings.<n>We find that gender cues often trigger larger distributional shifts than accent cues, indicating that current LALMs reproduce social stereotypes.
• Score: 50.73412962989565
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large Audio-Language Models (LALMs) are increasingly integrated into daily applications, yet their generative biases remain underexplored. Existing speech fairness benchmarks rely on synthetic speech and Multiple-Choice Questions (MCQs), both offering a fragmented view of fairness. We propose VIBE, a framework that evaluates generative bias through open-ended tasks such as personalized recommendations, using real-world human recordings. Unlike MCQs, our method allows stereotypical associations to manifest organically without predefined options, making it easily extensible to new tasks. Evaluating 11 state-of-the-art LALMs reveals systematic biases in realistic scenarios. We find that gender cues often trigger larger distributional shifts than accent cues, indicating that current LALMs reproduce social stereotypes.

Related papers

• GlobeAudio: A Multilingual Multicultural Benchmark for Naturalistic Evaluation of Large Audio-Language Models [15.48775980389557]
  Large Audio-Language Models (LALMs) integrate audio perception and language understanding within a unified framework.<n>Despite recent advances, evaluation for LALMs remains heavily underspecified relative to real-world requirements.<n>We propose GlobeAudio, a benchmark designed to evaluate naturalistic audio understanding.
  arXiv  Detail & Related papers  (2026-06-06T14:24:05Z)
• VoxEmo: Benchmarking Speech Emotion Recognition with Speech LLMs [54.75016325571445]
  Speech Large Language Models (LLMs) show great promise for speech emotion recognition via generative interfaces.<n> shifting from closed-set to open text generation introduces zero-shotity, making evaluation highly sensitive to prompts.<n>We present VoxEmo, a comprehensive SER benchmark encompassing 35 emotion corpora across 15 languages for Speech LLMs.
  arXiv  Detail & Related papers  (2026-03-09T21:10:34Z)
• Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts [29.864293711943038]
  We propose a framework for detecting stereotype-inducing words and attributing neuron-level bias in large language models.<n>Our framework first identifies stereotype-inducing adjectives and nouns via comparative analysis across demographic groups.<n> Experiments on three widely used LLMs demonstrate that our method effectively reduces bias while preserving overall model performance.
  arXiv  Detail & Related papers  (2026-02-04T10:27:36Z)
• A Comprehensive Study of Implicit and Explicit Biases in Large Language Models [1.0555164678638427]
  This study highlights the need to address biases in Large Language Models amid growing generative AI.<n>We studied bias-specific benchmarks such as StereoSet and CrowSPairs to evaluate the existence of various biases in multiple generative models such as BERT and GPT 3.5.<n>Results indicated fine-tuned models struggle with gender biases but excelled at identifying and avoiding racial biases.
  arXiv  Detail & Related papers  (2025-11-18T05:27:17Z)
• AHELM: A Holistic Evaluation of Audio-Language Models [78.20477815156484]
  multimodal audio-language models (ALMs) take interleaved audio and text as input and output text.<n>AHELM is a benchmark that aggregates various datasets -- including 2 new synthetic audio-text datasets called PARADE and CoRe-Bench.<n>We also standardize the prompts, inference parameters, and evaluation metrics to ensure equitable comparisons across models.
  arXiv  Detail & Related papers  (2025-08-29T07:40:39Z)
• WildSpeech-Bench: Benchmarking End-to-End SpeechLLMs in the Wild [40.210224623581155]
  Multi-modal Large Language Models (LLMs) have demonstrated strong capabilities of direct speech interaction.<n>Existing evaluation methods often adapt text-based benchmarks, overlooking speech's unique characteristics and challenges.<n>Here, we introduce the first comprehensive benchmark designed to systematically evaluate end-to-end speechLLMs in practical speech conversations.
  arXiv  Detail & Related papers  (2025-06-27T03:18:45Z)
• CO-VADA: A Confidence-Oriented Voice Augmentation Debiasing Approach for Fair Speech Emotion Recognition [49.27067541740956]
  We present CO-VADA, a Confidence-Oriented Voice Augmentation Debiasing Approach that mitigates bias without modifying model architecture or relying on demographic information.<n>CO-VADA identifies training samples that reflect bias patterns present in the training data and then applies voice conversion to alter irrelevant attributes and generate samples.<n>Our framework is compatible with various SER models and voice conversion tools, making it a scalable and practical solution for improving fairness in SER systems.
  arXiv  Detail & Related papers  (2025-06-06T13:25:56Z)
• MSA-ASR: Efficient Multilingual Speaker Attribution with frozen ASR Models [59.80042864360884]
  Speaker-attributed automatic speech recognition (SA-ASR) aims to transcribe speech while assigning transcripts to the corresponding speakers accurately.<n>This paper introduces a novel approach, leveraging a frozen multilingual ASR model to incorporate speaker attribution into the transcriptions.
  arXiv  Detail & Related papers  (2024-11-27T09:01:08Z)
• Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models [50.40276881893513]
  This study introduces Spoken Stereoset, a dataset specifically designed to evaluate social biases in Speech Large Language Models (SLLMs) By examining how different models respond to speech from diverse demographic groups, we aim to identify these biases. The findings indicate that while most models show minimal bias, some still exhibit slightly stereotypical or anti-stereotypical tendencies.
  arXiv  Detail & Related papers  (2024-08-14T16:55:06Z)
• Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models [38.64792118903994]
  We evaluate gender bias in SILLMs across four semantic-related tasks. Our analysis reveals that bias levels are language-dependent and vary with different evaluation methods.
  arXiv  Detail & Related papers  (2024-07-09T15:35:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.