Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception

• URL: http://arxiv.org/abs/2609.15215v1
• Date: Mon, 14 Sep 2026 08:35:35 GMT
• Title: Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception
• Abstract summary: Large Audio-Language Models (LALMs) have substantially advanced general audio understanding, yet they remain limited in fine-grained temporal perception.<n>We augment the LALM with a dedicated frame-level grounding model while leveraging its semantic modeling capability to represent the event query.<n>Experiments across diverse temporal grounding benchmarks demonstrate strong and consistent improvements over existing methods.
• Score: 26.919697203335343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Audio-Language Models (LALMs) have substantially advanced general audio understanding, yet they remain limited in fine-grained temporal perception, particularly in precise event localization. Existing approaches primarily post-train LALMs to predict event boundaries as timestamp tokens. However, this generative formulation lacks explicit correspondence between the timestamp predictions and fine-grained acoustic evidence, limiting the precision and reliability of temporal localization. To address this issue, we augment the LALM with a dedicated frame-level grounding model while leveraging its semantic modeling capability to represent the event query. Specifically, the frozen LALM encodes the event query with audio as context, and the grounding model combines these query representations with fine-grained audio features to localize the target event at the frame level. Extensive experiments across diverse temporal grounding benchmarks demonstrate strong and consistent improvements over existing methods. Further evaluation shows that the grounding model can provide temporal evidence to support downstream reasoning.

Related papers

• Auto-AEG: Scalable Data Construction for Open-Vocabulary Audio Event Grounding [36.43390300751778]
  We introduce AutoAEG, a scalable pipeline that constructs openvocabulary onset/offset supervision by automatic data construction and model finetuning.<n>Our results show that automatically constructed data, coupled with interval-aware reward function design, is an effective data-side route to expanding the temporal localization capability of LALMs.
  arXiv  Detail & Related papers  (2026-07-05T16:22:14Z)
• Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models [56.91801348360746]
  Large audio-language models (LALMs) generalize across speech, sound, and music.<n> unified decoders can exhibit a emphtemporal smoothing bias<n>We propose emphTemporal Contrastive Decoding (TCD), a training-free decoding method for unified LALMs.
  arXiv  Detail & Related papers  (2026-04-16T02:30:41Z)
• Towards Fine-grained Temporal Perception: Post-Training Large Audio-Language Models with Audio-Side Time Prompt [26.402383197553778]
  We propose Audio-Side Time Prompt and leverage Reinforcement Learning to develop the TimePro-RL framework for fine-grained temporal perception.<n>Specifically, we encodes as embedding timestamps and interleave them within the audio feature sequence as temporal coordinates to prompt the model.<n>Experiments demonstrate that TimePro-RL achieves significant performance gains across a range of audio temporal tasks, such as audio grounding, sound event detection, and dense audio captioning.
  arXiv  Detail & Related papers  (2026-04-15T10:50:29Z)
• In-Sync: Adaptation of Speech Aware Large Language Models for ASR with Word Level Timestamp Predictions [51.236148875603185]
  We extend an existing speech-aware language model to predict timestamps directly alongside transcripts.<n>We introduce a set of novel lightweight training strategies that improve alignment robustness while preserving recognition quality.
  arXiv  Detail & Related papers  (2026-04-14T20:56:24Z)
• LongAudio-RAG: Event-Grounded Question Answering over Multi-Hour Long Audio [6.935416517354558]
  LongAudio Generation (LARAG) is a framework that grounds Large Language Model (LLM) outputs in retrieved, timestamped acoustic event detections.<n>We demonstrate the practicality of our approach by deploying it in a hybrid edge-cloud environment.<n>Experiments show that structured, event-level retrieval significantly improves accuracy compared to vanilla Retrieval-Augmentment.
  arXiv  Detail & Related papers  (2026-02-16T10:15:22Z)
• Frame-Level Internal Tool Use for Temporal Grounding in Audio LMs [48.50855715191533]
  Large audio language models are increasingly used for complex audio understanding tasks.<n>They struggle with temporal tasks that require precise temporal grounding, such as word alignment and speaker diarization.<n>We propose frame-level internal tool use, a method that trains audio LMs to use their own internal audio representations to perform temporal grounding directly.
  arXiv  Detail & Related papers  (2026-02-10T19:19:52Z)
• Not in Sync: Unveiling Temporal Bias in Audio Chat Models [59.146710538620816]
  Large Audio Language Models (LALMs) are increasingly applied to audio understanding and multimodal reasoning.<n>We present the first systematic study of temporal bias in LALMs, revealing a key limitation in their timestamp prediction.
  arXiv  Detail & Related papers  (2025-10-14T06:29:40Z)
• TimeRefine: Temporal Grounding with Time Refining Video LLM [75.99665302872901]
  Video temporal grounding aims to localize relevant temporal boundaries in a video given a textual prompt.<n>We reformulate the temporal grounding task as a temporal refining task.<n>We incorporate an auxiliary prediction head that penalizes the model more if a predicted segment deviates further from the ground truth.
  arXiv  Detail & Related papers  (2024-12-12T18:59:11Z)
• Leveraging Language Model Capabilities for Sound Event Detection [10.792576135806623]
  We propose an end-to-end framework for understanding audio features while simultaneously generating sound event and their temporal location. Specifically, we employ pretrained acoustic models to capture discriminative features across different categories and language models for autoregressive text generation.
  arXiv  Detail & Related papers  (2023-08-22T15:59:06Z)
• DiffSED: Sound Event Detection with Denoising Diffusion [70.18051526555512]
  We reformulate the SED problem by taking a generative learning perspective. Specifically, we aim to generate sound temporal boundaries from noisy proposals in a denoising diffusion process. During training, our model learns to reverse the noising process by converting noisy latent queries to the groundtruth versions.
  arXiv  Detail & Related papers  (2023-08-14T17:29:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.