Aligning Audio Captions with Human Preferences

• URL: http://arxiv.org/abs/2509.14659v1
• Date: Thu, 18 Sep 2025 06:33:44 GMT
• Title: Aligning Audio Captions with Human Preferences
• Abstract summary: We propose a preference-aligned audio captioning framework based on Reinforcement Learning from Human Feedback.<n>We train a Contrastive Language-Audio Pretraining (CLAP)-based reward model using human-labeled pairwise preference data.<n>Our framework achieves performance comparable to supervised approaches with ground-truth data, demonstrating its effectiveness in aligning audio captioning with human preferences.
• Score: 7.207308857839664
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current audio captioning systems rely heavily on supervised learning with paired audio-caption datasets, which are expensive to curate and may not reflect human preferences in real-world scenarios. To address this limitation, we propose a preference-aligned audio captioning framework based on Reinforcement Learning from Human Feedback (RLHF). To effectively capture nuanced human preferences, we train a Contrastive Language-Audio Pretraining (CLAP)-based reward model using human-labeled pairwise preference data. This reward model is integrated into a reinforcement learning framework to fine-tune any baseline captioning system without relying on ground-truth caption annotations. Extensive human evaluations across multiple datasets show that our method produces captions preferred over those from baseline models, particularly in cases where the baseline models fail to provide correct and natural captions. Furthermore, our framework achieves performance comparable to supervised approaches with ground-truth data, demonstrating its effectiveness in aligning audio captioning with human preferences and its scalability in real-world scenarios.

Related papers

• VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation [79.34043102155964]
  We propose VA-Judger, a chain-of-thought omni-reward model for joint video-audio generation.<n>We show VA-Judger outperforms metric baselines in predicting human preferences on both in-domain and out-of-domain evaluations.
  arXiv  Detail & Related papers  (2026-08-19T06:53:14Z)
• Population-Aligned Audio Reproduction With LLM-Based Equalizers [12.358415132192148]
  We introduce a Large Language Model (LLM)-based alternative that maps natural language text prompts to equalization settings.<n>Our models exploit in-context learning and parameter-efficient fine-tuning techniques to reliably align with population-preferred equalization settings.<n>These results indicate that LLMs could function as "artificial equalizers," contributing to the development of more accessible, context-aware, and expert-level audio tuning methods.
  arXiv  Detail & Related papers  (2026-01-14T12:51:36Z)
• Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation [30.42124709340273]
  We identify three key barriers: limited large-scale audio-text corpora, insufficient caption diversity, and lack of systematic exploration and evaluation.<n>Our results demonstrate that audio-language pretraining yields competitive, transferable representations.<n>These findings establish audio-language pretraining as a viable pathway toward general-purpose audio representations.
  arXiv  Detail & Related papers  (2025-11-20T19:17:35Z)
• Incorporating Linguistic Constraints from External Knowledge Source for Audio-Visual Target Speech Extraction [87.49303116989708]
  We explore the potential of pre-trained speech-language models (PSLMs) and pre-trained language models (PLMs) as auxiliary knowledge sources for AV-TSE.<n>In this study, we propose incorporating the linguistic constraints from PSLMs or PLMs for the AV-TSE model as additional supervision signals.<n>Without any extra computational cost during inference, the proposed approach consistently improves speech quality and intelligibility.
  arXiv  Detail & Related papers  (2025-06-11T14:36:26Z)
• From Alignment to Advancement: Bootstrapping Audio-Language Alignment with Synthetic Data [55.2480439325792]
  Audio-aware large language models (ALLMs) have recently made great strides in understanding and processing audio inputs.<n>These models are typically adapted from text-based large language models (LLMs) through additional training on audio-related tasks.<n>We propose a data generation framework that produces contrastive-like training data, designed to enhance ALLMs' ability to differentiate between present and absent sounds.
  arXiv  Detail & Related papers  (2025-05-26T16:08:41Z)
• Classifier-Guided Captioning Across Modalities [69.75111271002137]
  We introduce a method to adapt captioning networks to the semantics of alternative settings, such as capturing audibility in audio captioning.<n>Our framework consists of two main components: (i) a frozen captioning system incorporating a language model (LM), and (ii) a text classifier that guides the captioning system.<n> Notably, when combined with an existing zero-shot audio captioning system, our framework improves its quality and sets state-of-the-art performance in zero-shot audio captioning.
  arXiv  Detail & Related papers  (2025-01-03T18:09:26Z)
• Can CLIP Help Sound Source Localization? [19.370071553914954]
  We introduce a framework that translates audio signals into tokens compatible with CLIP's text encoder. By directly using these embeddings, our method generates audio-grounded masks for the provided audio. Our findings suggest that utilizing pre-trained image-text models enable our model to generate more complete and compact localization maps for the sounding objects.
  arXiv  Detail & Related papers  (2023-11-07T15:26:57Z)
• CLIPSonic: Text-to-Audio Synthesis with Unlabeled Videos and Pretrained Language-Vision Models [50.42886595228255]
  We propose to learn the desired text-audio correspondence by leveraging the visual modality as a bridge. We train a conditional diffusion model to generate the audio track of a video, given a video frame encoded by a pretrained contrastive language-image pretraining model.
  arXiv  Detail & Related papers  (2023-06-16T05:42:01Z)
• Zero-shot text-to-speech synthesis conditioned using self-supervised speech representation model [13.572330725278066]
  A novel point of the proposed method is the direct use of the SSL model to obtain embedding vectors from speech representations trained with a large amount of data. The disentangled embeddings will enable us to achieve better reproduction performance for unseen speakers and rhythm transfer conditioned by different speeches.
  arXiv  Detail & Related papers  (2023-04-24T10:15:58Z)
• Language-Guided Audio-Visual Source Separation via Trimodal Consistency [64.0580750128049]
  A key challenge in this task is learning to associate the linguistic description of a sound-emitting object to its visual features and the corresponding components of the audio waveform. We adapt off-the-shelf vision-language foundation models to provide pseudo-target supervision via two novel loss functions. We demonstrate the effectiveness of our self-supervised approach on three audio-visual separation datasets.
  arXiv  Detail & Related papers  (2023-03-28T22:45:40Z)
• Matching Text and Audio Embeddings: Exploring Transfer-learning Strategies for Language-based Audio Retrieval [11.161404854726348]
  We present an analysis of large-scale pretrained deep learning models used for cross-modal (text-to-audio) retrieval. We use embeddings extracted by these models in a metric learning framework to connect matching pairs of audio and text.
  arXiv  Detail & Related papers  (2022-10-06T11:45:14Z)
• Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets [56.018551958004814]
  This paper addresses the task of generating fluent descriptions by training on a non-uniform combination of data sources. Large-scale datasets with noisy image-text pairs provide a sub-optimal source of supervision. We propose to leverage and separate semantics and descriptive style through the incorporation of a style token and keywords extracted through a retrieval component.
  arXiv  Detail & Related papers  (2021-11-24T19:00:05Z)
• Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval [28.57294189207084]
  The goal of audio captioning is to translate input audio into its description using natural language. The proposed method has succeeded to use a pre-trained language model for audio captioning. The oracle performance of the pre-trained model-based caption generator was clearly better than that of the conventional method trained from scratch.
  arXiv  Detail & Related papers  (2020-12-14T08:27:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.