Audio Question Answering with GRPO-Based Fine-Tuning and Calibrated Segment-Level Predictions

• URL: http://arxiv.org/abs/2511.14307v1
• Date: Tue, 18 Nov 2025 10:05:36 GMT
• Title: Audio Question Answering with GRPO-Based Fine-Tuning and Calibrated Segment-Level Predictions
• Authors: Marcel Gibier, Nolwenn Celton, Raphaël Duroselle, Pierre Serrano, Olivier Boeffard, Jean-François Bonastre,
• Abstract summary: We describe our submission to the DCASE 2025 Challenge for the task of Audio Question Answering(AQA)<n>Our system leverages the SSL backbone BEATs to extract frame-level audio features, which are then processed by a classification head to generate segment-level predictions of acoustic events.<n>These segment-level predictions are subsequently calibrated before producing event-level predictions.<n>Our method achieves an accuracy of 62.6 % on the development set, demonstrating the effectiveness of combining acoustic event reasoning with instruction-tuned large language models for AQA.
• Score: 5.960020042403666
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this report, we describe our submission to Track 5 of the DCASE 2025 Challenge for the task of Audio Question Answering(AQA). Our system leverages the SSL backbone BEATs to extract frame-level audio features, which are then processed by a classification head to generate segment-level predictions of acoustic events, following the Audioset ontology. These segment-level predictions are subsequently calibrated before producing event-level predictions. Finally, these predictions are incorporated into a structured prompt, along with the question and candidate answers. This prompt is then fed to a fine-tuned version of Qwen2.5-7B-Instruct, trained using the GRPO algorithm with a simple reward function. Our method achieves an accuracy of 62.6 % on the development set, demonstrating the effectiveness of combining acoustic event reasoning with instruction-tuned large language models for AQA.

Related papers

• Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decoding [25.824708878012753]
  Plan-Critic is a lightweight auxiliary model trained with aGAE-inspired objective to predict final instruction-following quality from partial generations.<n>Plan-Critic achieves up to a 10-point improvement in CLAP score over the AR baseline.<n>This work bridges the gap between causal generation and global semantic alignment, demonstrating that even strictly autoregressive models can plan ahead.
  arXiv  Detail & Related papers  (2026-01-18T07:26:23Z)
• Detect Any Sound: Open-Vocabulary Sound Event Detection with Multi-Modal Queries [18.147981850263708]
  We propose a query-based framework for open-vocabulary SED guided by multi-modal queries.<n>DASM formulates SED as a frame-level retrieval task, where audio features are matched against query vectors from text or audio prompts.<n>DASM effectively balances localization accuracy with generalization to novel classes, outperforming CLAP-based methods in open-vocabulary setting.
  arXiv  Detail & Related papers  (2025-07-22T08:24:01Z)
• Multi-Domain Audio Question Answering Toward Acoustic Content Reasoning in The DCASE 2025 Challenge [102.84031769492708]
  This task defines three QA subsets to test audio-language models on interactive question-answering over diverse acoustic scenes.<n>Preliminary results on the development set are compared, showing strong variation across models and subsets.<n>This challenge aims to advance the audio understanding and reasoning capabilities of audio-language models toward human-level acuity.
  arXiv  Detail & Related papers  (2025-05-12T09:04:16Z)
• Efficient Autoregressive Audio Modeling via Next-Scale Prediction [52.663934477127405]
  We analyze the token length of audio tokenization and propose a novel textbfScale-level textbfAudio textbfTokenizer (SAT)<n>Based on SAT, a scale-level textbfAcoustic textbfAutotextbfRegressive (AAR) modeling framework is proposed, which shifts the next-token AR prediction to next-scale AR prediction.
  arXiv  Detail & Related papers  (2024-08-16T21:48:53Z)
• Multitask frame-level learning for few-shot sound event detection [46.32294691870714]
  This paper focuses on few-shot Sound Event Detection (SED), which aims to automatically recognize and classify sound events with limited samples. We introduce an innovative multitask frame-level SED framework and TimeFilterAug, a linear timing mask for data augmentation. The proposed method achieves a F-score of 63.8%, securing the 1st rank in the few-shot bioacoustic event detection category.
  arXiv  Detail & Related papers  (2024-03-17T05:00:40Z)
• AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension [95.8442896569132]
  We introduce AIR-Bench, the first benchmark to evaluate the ability of Large Audio-Language Models (LALMs) to understand various types of audio signals and interact with humans in the textual format. Results demonstrate a high level of consistency between GPT-4-based evaluation and human evaluation.
  arXiv  Detail & Related papers  (2024-02-12T15:41:22Z)
• Segment-level Metric Learning for Few-shot Bioacoustic Event Detection [56.59107110017436]
  We propose a segment-level few-shot learning framework that utilizes both the positive and negative events during model optimization. Our system achieves an F-measure of 62.73 on the DCASE 2022 challenge task 5 (DCASE2022-T5) validation set, outperforming the performance of the baseline prototypical network 34.02 by a large margin.
  arXiv  Detail & Related papers  (2022-07-15T22:41:30Z)
• Speaker-Conditioned Hierarchical Modeling for Automated Speech Scoring [60.55025339250815]
  We propose a novel deep learning technique for non-native ASS, called speaker-conditioned hierarchical modeling. We take advantage of the fact that oral proficiency tests rate multiple responses for a candidate. In our technique, we take advantage of the fact that oral proficiency tests rate multiple responses for a candidate. We extract context from these responses and feed them as additional speaker-specific context to our network to score a particular response.
  arXiv  Detail & Related papers  (2021-08-30T07:00:28Z)
• Acoustic Scene Classification Using Bilinear Pooling on Time-liked and Frequency-liked Convolution Neural Network [4.131608702779222]
  This paper explores the use of harmonic and percussive source separation (HPSS) to split the audio into harmonic audio and percussive audio. The deep features extracted from these 2 CNNs will then be combined using bilinear pooling. The model is being evaluated on DCASE 2019 sub task 1a dataset and scored an average of 65% on development dataset.
  arXiv  Detail & Related papers  (2020-02-14T04:06:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.