Auditory Intelligence: Understanding the World Through Sound

• URL: http://arxiv.org/abs/2508.07829v1
• Date: Mon, 11 Aug 2025 10:25:58 GMT
• Title: Auditory Intelligence: Understanding the World Through Sound
• Authors: Hyeonuk Nam,
• Abstract summary: I propose a conceptual reframing of auditory intelligence as a layered, situated process that encompasses perception, reasoning, and interaction.<n>I introduce four cognitively inspired task paradigms-ASPIRE, SODA, AUX, and AUGMENT-those structure auditory understanding across time-frequency pattern captioning, hierarchical event/scene description, causal explanation, and goal-driven interpretation.
• Score: 4.6684925321613076
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in auditory intelligence has yielded high-performing systems for sound event detection (SED), acoustic scene classification (ASC), automated audio captioning (AAC), and audio question answering (AQA). Yet these tasks remain largely constrained to surface-level recognition-capturing what happened but not why, what it implies, or how it unfolds in context. I propose a conceptual reframing of auditory intelligence as a layered, situated process that encompasses perception, reasoning, and interaction. To instantiate this view, I introduce four cognitively inspired task paradigms-ASPIRE, SODA, AUX, and AUGMENT-those structure auditory understanding across time-frequency pattern captioning, hierarchical event/scene description, causal explanation, and goal-driven interpretation, respectively. Together, these paradigms provide a roadmap toward more generalizable, explainable, and human-aligned auditory intelligence, and are intended to catalyze a broader discussion of what it means for machines to understand sound.

Related papers

• Bagpiper: Solving Open-Ended Audio Tasks via Rich Captions [84.73122243726775]
  Bagpiper is an 8B audio foundation model that interprets physical audio via rich captions.<n>During fine-tuning, Bagpiper adopts a caption-then-process workflow to solve diverse tasks without task-specific priors.<n>To the best of our knowledge, Bagpiper is among the first works that achieve unified understanding generation for general audio.
  arXiv  Detail & Related papers  (2026-02-05T02:20:07Z)
• WoW-Bench: Evaluating Fine-Grained Acoustic Perception in Audio-Language Models via Marine Mammal Vocalizations [67.6147632074449]
  We introduce the World-of-Whale benchmark (WoW-Bench) to evaluate low-level auditory perception and cognition using marine mammal vocalizations.<n>WoW-Bench is composed of a Perception benchmark for categorizing novel sounds and a Cognition benchmark, inspired by Bloom's taxonomy, to assess the abilities to remember, understand, apply, and analyze sound events.<n>Experiments with state-of-the-art LALMs show performance far below human levels, indicating a need for stronger auditory grounding in LALMs.
  arXiv  Detail & Related papers  (2025-08-28T16:29:46Z)
• SpeechIQ: Speech Intelligence Quotient Across Cognitive Levels in Voice Understanding Large Language Models [76.07833875692722]
  Speech-based Intelligence Quotient (SIQ) is a new form of human cognition-inspired evaluation pipeline for voice understanding large language models.<n>Our framework represents a first-of-its-kind intelligence examination that bridges cognitive principles with voice-oriented benchmarks.
  arXiv  Detail & Related papers  (2025-07-25T15:12:06Z)
• Beyond Classification: Towards Speech Emotion Reasoning with Multitask AudioLLMs [37.62433475609052]
  We develop a strategy to enhance emotion recognition by producing semantically aligned, evidence-grounded explanations.<n>We introduce a unified framework combining reasoning-augmented data supervision, dual-encoder architecture, and task-alternating training.<n> Experiments on IEMOCAP and MELD show that our approach not only improves emotion prediction accuracy but also enhances the coherence and evidential grounding of the generated responses.
  arXiv  Detail & Related papers  (2025-06-07T14:52:58Z)
• Multi-Domain Audio Question Answering Toward Acoustic Content Reasoning in The DCASE 2025 Challenge [102.84031769492708]
  This task defines three QA subsets to test audio-language models on interactive question-answering over diverse acoustic scenes.<n>Preliminary results on the development set are compared, showing strong variation across models and subsets.<n>This challenge aims to advance the audio understanding and reasoning capabilities of audio-language models toward human-level acuity.
  arXiv  Detail & Related papers  (2025-05-12T09:04:16Z)
• AAD-LLM: Neural Attention-Driven Auditory Scene Understanding [9.596626274863832]
  We present Auditory Attention-Driven LLM (AAD-LLM), a prototype system that integrates brain signals to infer listener attention.<n>AAD-LLM predicts the attended speaker from neural activity, then conditions response generation on this inferred attentional state.<n>We evaluate AAD-LLM on speaker description, speech transcription and extraction, and question answering in multitalker scenarios.
  arXiv  Detail & Related papers  (2025-02-24T03:06:45Z)
• Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning [55.2480439325792]
  Large audio-language models (LALMs) have shown impressive capabilities in understanding and reasoning about audio and speech information.<n>These models still face challenges, including hallucinating non-existent sound events, misidentifying the order of sound events, and incorrectly attributing sound sources.
  arXiv  Detail & Related papers  (2024-10-21T15:55:27Z)
• Single-word Auditory Attention Decoding Using Deep Learning Model [9.698931956476692]
  Identifying auditory attention by comparing auditory stimuli and corresponding brain responses, is known as auditory attention decoding (AAD) This paper presents a deep learning approach, based on EEGNet, to address this challenge.
  arXiv  Detail & Related papers  (2024-10-15T21:57:19Z)
• Probing the Information Encoded in Neural-based Acoustic Models of Automatic Speech Recognition Systems [7.207019635697126]
  This article aims to determine which and where information is located in an automatic speech recognition acoustic model (AM) Experiments are performed on speaker verification, acoustic environment classification, gender classification, tempo-distortion detection systems and speech sentiment/emotion identification. Analysis showed that neural-based AMs hold heterogeneous information that seems surprisingly uncorrelated with phoneme recognition.
  arXiv  Detail & Related papers  (2024-02-29T18:43:53Z)
• Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Network [28.661704280484457]
  We propose Word-level End-to-End Neural Diarization (WEEND) with auxiliary network. We find WEEND has the potential to deliver high quality diarized text.
  arXiv  Detail & Related papers  (2023-09-15T15:48:45Z)
• Introducing Semantics into Speech Encoders [91.37001512418111]
  We propose an unsupervised way of incorporating semantic information from large language models into self-supervised speech encoders without labeled audio transcriptions. Our approach achieves similar performance as supervised methods trained on over 100 hours of labeled audio transcripts.
  arXiv  Detail & Related papers  (2022-11-15T18:44:28Z)
• On the Impact of Speech Recognition Errors in Passage Retrieval for Spoken Question Answering [13.013751306590303]
  We study the robustness of lexical and dense retrievers against questions with synthetic ASR noise. We create a new dataset with questions voiced by human users and use their transcriptions to show that the retrieval performance can further degrade when dealing with natural ASR noise instead of synthetic ASR noise.
  arXiv  Detail & Related papers  (2022-09-26T18:29:36Z)
• Contextualized Attention-based Knowledge Transfer for Spoken Conversational Question Answering [63.72278693825945]
  Spoken conversational question answering (SCQA) requires machines to model complex dialogue flow. We propose CADNet, a novel contextualized attention-based distillation approach. We conduct extensive experiments on the Spoken-CoQA dataset and demonstrate that our approach achieves remarkable performance.
  arXiv  Detail & Related papers  (2020-10-21T15:17:18Z)
• Speaker-Utterance Dual Attention for Speaker and Utterance Verification [77.2346078109261]
  We implement an idea of speaker-utterance dual attention (SUDA) in a unified neural network. The proposed SUDA features an attention mask mechanism to learn the interaction between the speaker and utterance information streams.
  arXiv  Detail & Related papers  (2020-08-20T11:37:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.