WoW-Bench: Evaluating Fine-Grained Acoustic Perception in Audio-Language Models via Marine Mammal Vocalizations

• URL: http://arxiv.org/abs/2508.20976v1
• Date: Thu, 28 Aug 2025 16:29:46 GMT
• Title: WoW-Bench: Evaluating Fine-Grained Acoustic Perception in Audio-Language Models via Marine Mammal Vocalizations
• Authors: Jaeyeon Kim, Heeseung Yun, Sang Hoon Woo, Chao-Han Huck Yang, Gunhee Kim,
• Abstract summary: We introduce the World-of-Whale benchmark (WoW-Bench) to evaluate low-level auditory perception and cognition using marine mammal vocalizations.<n>WoW-Bench is composed of a Perception benchmark for categorizing novel sounds and a Cognition benchmark, inspired by Bloom's taxonomy, to assess the abilities to remember, understand, apply, and analyze sound events.<n>Experiments with state-of-the-art LALMs show performance far below human levels, indicating a need for stronger auditory grounding in LALMs.
• Score: 67.6147632074449
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large audio language models (LALMs) extend language understanding into the auditory domain, yet their ability to perform low-level listening, such as pitch and duration detection, remains underexplored. However, low-level listening is critical for real-world, out-of-distribution tasks where models must reason about unfamiliar sounds based on fine-grained acoustic cues. To address this gap, we introduce the World-of-Whale benchmark (WoW-Bench) to evaluate low-level auditory perception and cognition using marine mammal vocalizations. WoW-bench is composed of a Perception benchmark for categorizing novel sounds and a Cognition benchmark, inspired by Bloom's taxonomy, to assess the abilities to remember, understand, apply, and analyze sound events. For the Cognition benchmark, we additionally introduce distractor questions to evaluate whether models are truly solving problems through listening rather than relying on other heuristics. Experiments with state-of-the-art LALMs show performance far below human levels, indicating a need for stronger auditory grounding in LALMs.

Related papers

• Spatial Blind Spot: Auditory Motion Perception Deficits in Audio LLMs [39.209987830131816]
  Large Audio-Language Models (LALMs) have recently shown impressive progress in speech recognition, audio captioning, and auditory question answering.<n>Yet, whether these models can perceive dynamics, particularly the motion of sound sources, remains unclear.<n>We introduce AMPBench, the first benchmark explicitly designed to evaluate auditory motion understanding.
  arXiv  Detail & Related papers  (2025-11-17T11:45:41Z)
• Thinking While Listening: Simple Test Time Scaling For Audio Classification [61.3564313676731]
  We propose a framework that enables neural models to "think while listening" to everyday sounds, thereby enhancing audio classification performance.<n>Motivated by recent advances in the reasoning capabilities of large language models, we address two central questions: (i) how can thinking be incorporated into existing audio classification pipelines to enable reasoning in the category space and improve performance, and (ii) can a new architecture be designed from the ground up to support both thinking and test-time scaling.
  arXiv  Detail & Related papers  (2025-09-24T01:17:24Z)
• AuditoryBench++: Can Language Models Understand Auditory Knowledge without Hearing? [13.180643834705114]
  We present AuditoryBench++, a benchmark for evaluating auditory knowledge and reasoning in text-only settings.<n>The benchmark encompasses tasks that range from basic auditory comparisons to contextually grounded reasoning.<n>We also introduce AIR-CoT, a novel auditory imagination reasoning method that generates and integrates auditory information during inference.
  arXiv  Detail & Related papers  (2025-09-22T11:45:22Z)
• Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning [55.2480439325792]
  Large audio-language models (LALMs) have shown impressive capabilities in understanding and reasoning about audio and speech information.<n>These models still face challenges, including hallucinating non-existent sound events, misidentifying the order of sound events, and incorrectly attributing sound sources.
  arXiv  Detail & Related papers  (2024-10-21T15:55:27Z)
• Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Models [49.87432626548563]
  We introduce methods to assess the extent of object hallucination of publicly available LALMs. Our findings reveal that LALMs are comparable to specialized audio captioning models in their understanding of audio content. We explore the potential of prompt engineering to enhance LALMs' performance on discriminative questions.
  arXiv  Detail & Related papers  (2024-06-12T16:51:54Z)
• AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension [95.8442896569132]
  We introduce AIR-Bench, the first benchmark to evaluate the ability of Large Audio-Language Models (LALMs) to understand various types of audio signals and interact with humans in the textual format. Results demonstrate a high level of consistency between GPT-4-based evaluation and human evaluation.
  arXiv  Detail & Related papers  (2024-02-12T15:41:22Z)
• Self-supervised models of audio effectively explain human cortical responses to speech [71.57870452667369]
  We capitalize on the progress of self-supervised speech representation learning to create new state-of-the-art models of the human auditory system. We show that these results show that self-supervised models effectively capture the hierarchy of information relevant to different stages of speech processing in human cortex.
  arXiv  Detail & Related papers  (2022-05-27T22:04:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.