Listen, Reason, and Segment: Aligning LALMs with Editorial Judgment for Media Chapterization

• URL: http://arxiv.org/abs/2608.16539v1
• Date: Mon, 17 Aug 2026 13:14:01 GMT
• Title: Listen, Reason, and Segment: Aligning LALMs with Editorial Judgment for Media Chapterization
• Abstract summary: Large Audio Language Models (LALMs) have made rapid progress on standardized benchmarks, yet their deployment in practical media, curation, archival indexing, and content distribution remains largely unrealized.<n>We identify automated audio chapterization, the task of segmenting continuous audio streams into thematically coherent chapters.<n>We present AudioChaps, a framework for aligning end-to-end LALMs for this task.
• Score: 58.34381853842135
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Audio Language Models (LALMs) have made rapid progress on standardized benchmarks, yet their deployment in practical media workflows, curation, archival indexing, and content distribution remains largely unrealized. We identify automated audio chapterization, the task of segmenting continuous audio streams into thematically coherent chapters, as a demanding and commercially consequential setting that exposes this gap. Chapterization is challenging because boundaries are defined less by objective acoustic events than by subjective editorial judgment, requiring models to reason sequentially over long acoustic contexts and approximate creator-authored boundary decisions. We present AudioChaps, a post-training framework for aligning end-to-end LALMs for this task via Group Relative Policy Optimization (GRPO) guided by Chain-of-Thought (CoT) reasoning. To support training and evaluation, we curate three datasets: AudioChaps-Alignment, derived from creator-annotated chapter boundaries on YouTube; AudioChaps-CoT, which provides structured supervision for well-formatted, high-quality, and evidence-grounded boundary reasoning; and AudioChaps-Eval, a held-out benchmark for audio chapterization. Applying GRPO directly without a Supervised Fine-Tuning (SFT) cold start, AudioChaps-R1-Zero already improves average F1 by 33 points over the state-of-the-art LALM Audio-Flamingo-3-Think. The AudioChaps framework produces our final aligned LALM, AudioChaps-R1, which improves average F1 by 49 points. These results demonstrate that GRPO-trained LALMs can reliably transform unstructured auditory streams into navigable, structured media. Our code, models, and dataset resources will be released upon acceptance at https://github.com/ta012/AudioChaps.

Related papers

• TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models [58.80225070159013]
  Large audio-language models (LALMs) describe audio at the clip level but cannot assign timestamps to the events, speakers, or sounds they identify.<n>We present TEMPO, the first unified model to handle audio, speech, and music timestamping tasks.
  arXiv  Detail & Related papers  (2026-08-30T19:49:21Z)
• Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models [56.91801348360746]
  Large audio-language models (LALMs) generalize across speech, sound, and music.<n> unified decoders can exhibit a emphtemporal smoothing bias<n>We propose emphTemporal Contrastive Decoding (TCD), a training-free decoding method for unified LALMs.
  arXiv  Detail & Related papers  (2026-04-16T02:30:41Z)
• LAMB: LLM-based Audio Captioning with Modality Gap Bridging via Cauchy-Schwarz Divergence [35.123477091633866]
  LAMB is an audio captioning framework that bridges the modality gap between audio embeddings and the text embedding space.<n>A Cross-Modal Aligner minimizes Cauchy-Schwarz divergence while maximizing mutual information.<n>A Two-Stream Adapter that extracts semantically enriched audio embeddings delivers richer information to the Cross-Modal Aligner.
  arXiv  Detail & Related papers  (2026-01-08T07:05:35Z)
• SightSound-R1: Cross-Modal Reasoning Distillation from Vision to Audio Language Models [18.802543558300044]
  We present SightSound-R1, a cross-modal distillation framework that transfers advanced reasoning from a stronger LVLM teacher to a weaker LALM student.<n>Results show that SightSound-R1 improves LALM reasoning performance both in the in-domain AVQA test set and in unseen auditory scenes and questions.
  arXiv  Detail & Related papers  (2025-09-19T06:39:39Z)
• Detect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries [18.147981850263708]
  We propose a query-based framework for open-vocabulary SED guided by multi-modal queries.<n>DASM formulates SED as a frame-level retrieval task, where audio features are matched against query vectors from text or audio prompts.<n>DASM effectively balances localization accuracy with generalization to novel classes, outperforming CLAP-based methods in open-vocabulary setting.
  arXiv  Detail & Related papers  (2025-07-22T08:24:01Z)
• PAL: Probing Audio Encoders via LLMs - Audio Information Transfer into LLMs [29.049167884343998]
  Integration of audio perception into large language models (LLMs) is an emerging research area for enabling machine listening applications.<n>We propose an efficient alternative, Lightweight Audio LLM Integration (LAL)<n>LAL encodes rich audio semantics at an appropriate level of abstraction for integration into different blocks of LLMs.
  arXiv  Detail & Related papers  (2025-06-12T07:23:07Z)
• From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Data [55.2480439325792]
  Audio-aware large language models (ALLMs) have recently made great strides in understanding and processing audio inputs.<n>These models are typically adapted from text-based large language models (LLMs) through additional training on audio-related tasks.<n>We propose a data generation framework that produces contrastive-like training data, designed to enhance ALLMs' ability to differentiate between present and absent sounds.
  arXiv  Detail & Related papers  (2025-05-26T16:08:41Z)
• CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment [76.32508013503653]
  We propose CAV-MAE Sync as a simple yet effective extension of the original CAV-MAE framework for self-supervised audio-visual learning.<n>We tackle the mismatch between modalities by treating audio as a temporal sequence aligned with video frames, rather than using global representations.<n>We improve spatial localization by introducing learnable register tokens that reduce semantic load on patch tokens.
  arXiv  Detail & Related papers  (2025-05-02T12:59:58Z)
• C3LLM: Conditional Multimodal Content Generation Using Large Language Models [66.11184017840688]
  We introduce C3LLM, a novel framework combining three tasks of video-to-audio, audio-to-text, and text-to-audio together. C3LLM adapts the Large Language Model (LLM) structure as a bridge for aligning different modalities. Our method combines the previous tasks of audio understanding, video-to-audio generation, and text-to-audio generation together into one unified model.
  arXiv  Detail & Related papers  (2024-05-25T09:10:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.