Improving Text-to-Music Generation with Human Preference Rewards

• URL: http://arxiv.org/abs/2606.21670v1
• Date: Fri, 19 Jun 2026 18:22:24 GMT
• Title: Improving Text-to-Music Generation with Human Preference Rewards
• Abstract summary: We describe our entry to the efficiency track of the Academic Text-to-Music (ATTM) Grand Challenge at ICME 2026.<n>We add a learned human-preference reward from TuneJury, a twin pairwise ranker trained over open music-preference datasets.<n>The reward serves both as a training-time conditioning signal and as a sample-selection criterion.
• Score: 9.458042125925573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We describe our entry to the efficiency track of the Academic Text-to-Music (ATTM) Grand Challenge at ICME 2026. Beyond the challenge protocol's FAD-CLAP and CLAP score, we add a learned human-preference reward from TuneJury, a twin pairwise ranker trained over open music-preference datasets. The reward serves both as a training-time conditioning signal and as a sample-selection criterion. The pipeline combines five engineering decisions on a 120M-parameter FluxAudio-S backbone, four at training time and one at inference: (i) training-time reward conditioning that doubles as an inference-time CFG axis, (ii) a sweep over five score-conditioning architectures, where training and inference use different variants, (iii) expert iteration on the top decile, (iv) a short preference-tuning pass (CRPO) for audio-text alignment, and (v) inference post-processing via joint CFG, source separation, and loudness normalization. Per-stage decomposition on 100 Song Describer prompts shows training-time reward conditioning as a functional conditioning axis, expert iteration as the dominant contributor, the preference-tuning pass adding only noise-level gain, and the inference-time score scalar already saturated by the end of the chain.

Related papers

• AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing [74.97664372105693]
  We introduce AuK, an open-source foundational model that unifies speech generation and editing through a common interface of natural-language instructions and audio context.<n>We construct approximately 3.03 billion instruction--audio instances and 1.95 million hours of effective supervision across five task families.
  arXiv  Detail & Related papers  (2026-09-08T15:59:16Z)
• TuneJury: An Open Metric for Improving Music Generation Preference Alignment [33.898713000368794]
  TuneJury is a pairwise reward model for text-to-music.<n>It predicts a music preference score from a text prompt and an audio clip.<n>It generalizes to both held-out test pairs and out-of-distribution benchmarks.
  arXiv  Detail & Related papers  (2026-06-15T17:39:30Z)
• Making the Most of Limited Data: Score-Aware Training for Text-to-Music Generation [1.0323063834827415]
  State-of-the-art text-to-music generation systems rely on massive proprietary datasets and industrial-scale compute.<n>We propose textitscore-aware training, which treats audio-caption alignment score as a direct supervision signal throughout the pipeline.
  arXiv  Detail & Related papers  (2026-06-05T15:24:36Z)
• Representation-Regularized Convolutional Audio Transformer for Audio Understanding [53.092757178419355]
  bootstrapping representations from scratch is computationally expensive, often requiring extensive training to converge.<n>We propose the Convolutional Audio Transformer (CAT), a unified framework designed to address these challenges.
  arXiv  Detail & Related papers  (2026-01-29T12:16:19Z)
• PrismAudio: Decomposed Chain-of-Thoughts and Multi-dimensional Rewards for Video-to-Audio Generation [57.864929968616586]
  Video-to-Audio (V2A) generation requires balancing four critical perceptual dimensions.<n>We introduce PrismAudio, the first framework to integrate Reinforcement Learning into V2A generation with specialized Chain-of-Thought (CoT) planning.
  arXiv  Detail & Related papers  (2025-11-24T07:11:12Z)
• A Systematic Examination of Preference Learning through the Lens of Instruction-Following [83.71180850955679]
  We use a novel synthetic data generation pipeline to generate 48,000 instruction unique-following prompts.<n>With our synthetic prompts, we use two preference dataset curation methods - rejection sampling (RS) and Monte Carlo Tree Search (MCTS)<n>Experiments reveal that shared prefixes in preference pairs, as generated by MCTS, provide marginal but consistent improvements.<n>High-contrast preference pairs generally outperform low-contrast pairs; however, combining both often yields the best performance.
  arXiv  Detail & Related papers  (2024-12-18T15:38:39Z)
• End-to-End Real-World Polyphonic Piano Audio-to-Score Transcription with Hierarchical Decoding [4.604877755214193]
  Existing end-to-end piano A2S systems have been trained and evaluated with only synthetic data. We propose a sequence-to-sequence (Seq2Seq) model with a hierarchical decoder that aligns with the hierarchical structure of musical scores. We propose a two-stage training scheme, which involves pre-training the model using an expressive performance rendering system on synthetic audio, followed by fine-tuning the model using recordings of human performance.
  arXiv  Detail & Related papers  (2024-05-22T10:52:04Z)
• D4AM: A General Denoising Framework for Downstream Acoustic Models [45.04967351760919]
  Speech enhancement (SE) can be used as a front-end strategy to aid automatic speech recognition (ASR) systems. Existing training objectives of SE methods are not fully effective at integrating speech-text and noisy-clean paired data for training toward unseen ASR systems. We propose a general denoising framework, D4AM, for various downstream acoustic models.
  arXiv  Detail & Related papers  (2023-11-28T08:27:27Z)
• Low-complexity deep learning frameworks for acoustic scene classification [64.22762153453175]
  We present low-complexity deep learning frameworks for acoustic scene classification (ASC) The proposed frameworks can be separated into four main steps: Front-end spectrogram extraction, online data augmentation, back-end classification, and late fusion of predicted probabilities. Our experiments conducted on DCASE 2022 Task 1 Development dataset have fullfiled the requirement of low-complexity and achieved the best classification accuracy of 60.1%.
  arXiv  Detail & Related papers  (2022-06-13T11:41:39Z)
• Scenario Aware Speech Recognition: Advancements for Apollo Fearless Steps & CHiME-4 Corpora [70.46867541361982]
  We consider a general non-semantic speech representation, which is trained with a self-supervised criteria based on triplet loss called TRILL. We observe +5.42% and +3.18% relative WER improvement for the development and evaluation sets of Fearless Steps.
  arXiv  Detail & Related papers  (2021-09-23T00:43:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.