Towards Fine-grained Temporal Perception: Post-Training Large Audio-Language Models with Audio-Side Time Prompt

• URL: http://arxiv.org/abs/2604.13715v1
• Date: Wed, 15 Apr 2026 10:50:29 GMT
• Title: Towards Fine-grained Temporal Perception: Post-Training Large Audio-Language Models with Audio-Side Time Prompt
• Abstract summary: We propose Audio-Side Time Prompt and leverage Reinforcement Learning to develop the TimePro-RL framework for fine-grained temporal perception.<n>Specifically, we encodes as embedding timestamps and interleave them within the audio feature sequence as temporal coordinates to prompt the model.<n>Experiments demonstrate that TimePro-RL achieves significant performance gains across a range of audio temporal tasks, such as audio grounding, sound event detection, and dense audio captioning.
• Score: 26.402383197553778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Audio-Language Models (LALMs) enable general audio understanding and demonstrate remarkable performance across various audio tasks. However, these models still face challenges in temporal perception (e.g., inferring event onset and offset), leading to limited utility in fine-grained scenarios. To address this issue, we propose Audio-Side Time Prompt and leverage Reinforcement Learning (RL) to develop the TimePro-RL framework for fine-grained temporal perception. Specifically, we encode timestamps as embeddings and interleave them within the audio feature sequence as temporal coordinates to prompt the model. Furthermore, we introduce RL following Supervised Fine-Tuning (SFT) to directly optimize temporal alignment performance. Experiments demonstrate that TimePro-RL achieves significant performance gains across a range of audio temporal tasks, such as audio grounding, sound event detection, and dense audio captioning, validating its robust effectiveness.

Related papers

• Augmenting Large Audio-Language Models with Frame-Level Grounding for Fine-Grained Temporal Perception [26.919697203335343]
  Large Audio-Language Models (LALMs) have substantially advanced general audio understanding, yet they remain limited in fine-grained temporal perception.<n>We augment the LALM with a dedicated frame-level grounding model while leveraging its semantic modeling capability to represent the event query.<n>Experiments across diverse temporal grounding benchmarks demonstrate strong and consistent improvements over existing methods.
  arXiv  Detail & Related papers  (2026-09-14T08:35:35Z)
• TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models [58.80225070159013]
  Large audio-language models (LALMs) describe audio at the clip level but cannot assign timestamps to the events, speakers, or sounds they identify.<n>We present TEMPO, the first unified model to handle audio, speech, and music timestamping tasks.
  arXiv  Detail & Related papers  (2026-08-30T19:49:21Z)
• MOSS-Audio Technical Report [80.01042232102883]
  MOSS-Audio is a unified audio-language model for speech, environmental sound, and music understanding.<n>It supports audio captioning, time-aware question answering, timestamped transcription, and audio-grounded reasoning.
  arXiv  Detail & Related papers  (2026-06-01T07:19:22Z)
• AudioMosaic: Contrastive Masked Audio Representation Learning [53.52371029884106]
  We introduce textbfAudioMosaic, a contrastive learning-based audio encoder for general audio understanding.<n>AudioMosaic constructs positive pairs by applying structured time-frequency masking to spectrogram patches.<n>Experiments show that AudioMosaic achieves state-of-the-art performance on several standard audio benchmarks.
  arXiv  Detail & Related papers  (2026-05-14T00:56:51Z)
• Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models [56.91801348360746]
  Large audio-language models (LALMs) generalize across speech, sound, and music.<n> unified decoders can exhibit a emphtemporal smoothing bias<n>We propose emphTemporal Contrastive Decoding (TCD), a training-free decoding method for unified LALMs.
  arXiv  Detail & Related papers  (2026-04-16T02:30:41Z)
• Frame-Level Internal Tool Use for Temporal Grounding in Audio LMs [48.50855715191533]
  Large audio language models are increasingly used for complex audio understanding tasks.<n>They struggle with temporal tasks that require precise temporal grounding, such as word alignment and speaker diarization.<n>We propose frame-level internal tool use, a method that trains audio LMs to use their own internal audio representations to perform temporal grounding directly.
  arXiv  Detail & Related papers  (2026-02-10T19:19:52Z)
• Not in Sync: Unveiling Temporal Bias in Audio Chat Models [59.146710538620816]
  Large Audio Language Models (LALMs) are increasingly applied to audio understanding and multimodal reasoning.<n>We present the first systematic study of temporal bias in LALMs, revealing a key limitation in their timestamp prediction.
  arXiv  Detail & Related papers  (2025-10-14T06:29:40Z)
• AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs [53.248502396225724]
  AudioMarathon is a benchmark designed to evaluate both understanding and inference efficiency on long-form audio.<n>We evaluate state-of-the-art LALMs and observe clear performance drops as audio length grows.<n>The results show large gaps across current LALMs and highlight the need for better temporal reasoning.
  arXiv  Detail & Related papers  (2025-10-08T17:50:16Z)
• Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning [55.2480439325792]
  Large audio-language models (LALMs) have shown impressive capabilities in understanding and reasoning about audio and speech information.<n>These models still face challenges, including hallucinating non-existent sound events, misidentifying the order of sound events, and incorrectly attributing sound sources.
  arXiv  Detail & Related papers  (2024-10-21T15:55:27Z)
• Enhancing Temporal Understanding in Audio Question Answering for Large Audio Language Models [0.9285295512807729]
  The Audio Question Answering (AQA) task includes audio event classification, audio captioning, and open-ended reasoning.<n>LALMs excel in general audio understanding, but are limited in temporal reasoning.<n>This paper addresses these challenges and limitations in audio temporal reasoning.
  arXiv  Detail & Related papers  (2024-09-10T05:26:53Z)
• Tango 2: Aligning Diffusion-based Text-to-Audio Generations through Direct Preference Optimization [70.13218512896032]
  Generation of audio from text prompts is an important aspect of such processes in the music and film industry. Our hypothesis is focusing on how these aspects of audio generation could improve audio generation performance in the presence of limited data. We synthetically create a preference dataset where each prompt has a winner audio output and some loser audio outputs for the diffusion model to learn from.
  arXiv  Detail & Related papers  (2024-04-15T17:31:22Z)
• Leveraging Language Model Capabilities for Sound Event Detection [10.792576135806623]
  We propose an end-to-end framework for understanding audio features while simultaneously generating sound event and their temporal location. Specifically, we employ pretrained acoustic models to capture discriminative features across different categories and language models for autoregressive text generation.
  arXiv  Detail & Related papers  (2023-08-22T15:59:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.