Locality Matters for Training-Free Audio Token Compression in Audio-Language Models

• URL: http://arxiv.org/abs/2605.25179v1
• Date: Sun, 24 May 2026 17:19:31 GMT
• Title: Locality Matters for Training-Free Audio Token Compression in Audio-Language Models
• Abstract summary: Local Temporal Bipartite Merging is a training-free encoder-space compression method that merges similar nearby audio tokens under an explicit temporal window constraint.<n>We show that locality-aware merging is more favorable for captioning at several compression settings, especially under stronger compression.
• Score: 10.910678708402651
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Audio-language models (ALMs) are increasingly used for audio captioning, question answering, and open-ended audio understanding, but their inference cost remains high when audio inputs are represented as long prefix-token sequences. These audio prefixes consume context budget, increase memory usage, and make deployment harder in resource-constrained or latency-sensitive settings. Existing training-free audio-token reduction methods mainly rely on fixed pooling or score-based pruning. Fixed pooling is content-agnostic, while score-based pruning can preserve isolated salient tokens but discard nearby acoustic context. We propose Local Temporal Bipartite Merging (LTBM), a training-free encoder-space compression method that merges similar nearby audio tokens under an explicit temporal window constraint. Beyond introducing LTBM, we use a controlled Global Merge variant to isolate whether temporal locality itself is a useful inductive bias for audio-token compression. Experiments on AudioCaps, Clotho, and MMAU with Qwen2-Audio show evidence of a task-dependent locality effect: locality-aware merging is more favorable for captioning at several compression settings, especially under stronger compression, while global matching is more competitive for multiple-choice audio understanding. A cross-backbone validation on Audio Flamingo 3 further supports the captioning-side advantage of locality-aware merging under moderate and aggressive compression.

Related papers

• VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference [53.883871460875234]
  VoxZip is a semantic-anchored KV cache compression framework.<n>First stage uses automatic speech recognition (ASR) transcriptions as explicit semantic anchors to temporally align, compress, and fuse audio tokens.<n>Second stage employs a dynamic filtering strategy based on temporally decayed accumulated attention to evict non-essential tokens.
  arXiv  Detail & Related papers  (2026-08-09T08:17:18Z)
• Acoustic Cue Alignment in Audio Language Models for Speech Emotion Recognition [58.25449304752214]
  We study whether explicit acoustic cues are grounded when the raw audio is already available.<n>We deriving six interpretable acoustic concept tokens from the standardised eGeMAPS paralinguistic feature set.<n>Aligned tokens improve unweighted average recall (UAR), whereas shuffled, conflicting, or corrupted tokens reduce performance.<n>We argue that token-only interventions provide a practical way to probe audio-grounded cue use, robustness, and interpretability in ALM-based affective computing.
  arXiv  Detail & Related papers  (2026-06-05T14:26:06Z)
• EntangleCodec: A Unified Discrete Audio Tokenizer via Semantic-Acoustic Entanglement [54.95940885045993]
  EntangleCodec is a unified discrete audio tokenizer that learns caption-aligned semantic-acoustic representations before quantization.<n>It captures linguistic content, speaker identity, emotion, prosody, and acoustic scenes within a compact token stream.<n>It achieves reconstruction quality competitive with specialized codecs, and supports both TTS and TTA generation in a unified framework.
  arXiv  Detail & Related papers  (2026-06-01T18:05:18Z)
• MOSS-Audio Technical Report [80.01042232102883]
  MOSS-Audio is a unified audio-language model for speech, environmental sound, and music understanding.<n>It supports audio captioning, time-aware question answering, timestamped transcription, and audio-grounded reasoning.
  arXiv  Detail & Related papers  (2026-06-01T07:19:22Z)
• AudioMosaic: Contrastive Masked Audio Representation Learning [53.52371029884106]
  We introduce textbfAudioMosaic, a contrastive learning-based audio encoder for general audio understanding.<n>AudioMosaic constructs positive pairs by applying structured time-frequency masking to spectrogram patches.<n>Experiments show that AudioMosaic achieves state-of-the-art performance on several standard audio benchmarks.
  arXiv  Detail & Related papers  (2026-05-14T00:56:51Z)
• Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models [56.91801348360746]
  Large audio-language models (LALMs) generalize across speech, sound, and music.<n> unified decoders can exhibit a emphtemporal smoothing bias<n>We propose emphTemporal Contrastive Decoding (TCD), a training-free decoding method for unified LALMs.
  arXiv  Detail & Related papers  (2026-04-16T02:30:41Z)
• DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression [5.201210378318278]
  We propose Dynamic Audio-driven Semantic cHunking (DASH) to align token compression with semantic structure.<n>DASH treats audio embeddings as a semantic anchor and detects boundary candidates via cosine-similarity discontinuities.<n> experiments on AVUT, VideoMME, and WorldSense demonstrate that DASH maintains superior accuracy while achieving higher compression ratios compared to prior methods.
  arXiv  Detail & Related papers  (2026-03-15T15:22:06Z)
• EchoingPixels: Cross-Modal Adaptive Token Reduction for Efficient Audio-Visual LLMs [28.295585578439212]
  EchoingPixels is a framework inspired by the coexistence and interaction of visuals and sound in real-world scenes.<n>It reduces tokens from an entire combined pool of audio-visual tokens rather than using fixed budgets per modality.<n>It achieves performance comparable to strong baselines using only 5-20% of the original tokens, with a 2-3x speedup and memory reduction.
  arXiv  Detail & Related papers  (2025-12-11T06:18:58Z)
• OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models [21.542205813984243]
  We present a training-free, audio-visual token-compression framework that optimize multimodal token representation and inference.<n>Specifically, OmniZip first identifies salient audio tokens, then computes an audio retention score for each time group to capture information.<n>For each time window, OmniZip compresses the video tokens using an interleaved-temporal scheme.
  arXiv  Detail & Related papers  (2025-11-18T15:22:32Z)
• SecoustiCodec: Cross-Modal Aligned Streaming Single-Codecbook Speech Codec [83.61175662066364]
  Speech codecs serve as a crucial bridge in unifying speech and text language models.<n>Existing methods face several challenges in semantic encoding.<n>We propose SecoustiCodec, a cross-modal aligned low-bitrate streaming speech codecs.
  arXiv  Detail & Related papers  (2025-08-04T19:22:14Z)
• Separate Anything You Describe [53.30484933564858]
  Language-queried audio source separation (LASS) is a new paradigm for computational auditory scene analysis (CASA)<n>AudioSep is a foundation model for open-domain audio source separation with natural language queries.
  arXiv  Detail & Related papers  (2023-08-09T16:09:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.