TuneJury: An Open Metric for Improving Music Generation Preference Alignment

• URL: http://arxiv.org/abs/2606.17006v1
• Date: Mon, 15 Jun 2026 17:39:30 GMT
• Title: TuneJury: An Open Metric for Improving Music Generation Preference Alignment
• Abstract summary: TuneJury is a pairwise reward model for text-to-music.<n>It predicts a music preference score from a text prompt and an audio clip.<n>It generalizes to both held-out test pairs and out-of-distribution benchmarks.
• Score: 33.898713000368794
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce TuneJury, an open, instance-level pairwise reward model for text-to-music that predicts a music preference score from a text prompt and an audio clip. The released checkpoint is trained on publicly available human-preference labels covering arena-style (A vs. B) votes, metric-alignment preference pairs, crowdsourced pairwise comparisons, and expert aesthetic ratings. The predicted score margin between two clips is well calibrated on our held-out test split, supporting data filtering via a simple score threshold. TuneJury generalizes to both held-out test pairs and out-of-distribution benchmarks, remaining competitive with prior baselines on the latter. For generators released after training, we introduce anchor calibration, a post-hoc, per-system Bradley-Terry calibration that recovers agreement at substantially better data efficiency than from-scratch retraining. The same frozen reward drives consistent reward-axis gains across three downstream applications: inference-time best-of-N selection, DITTO-style latent optimization, and expert-iteration post-training. TuneJury is available at https://github.com/yonghyunk1m/TuneJury.

Related papers

• VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation [79.34043102155964]
  We propose VA-Judger, a chain-of-thought omni-reward model for joint video-audio generation.<n>We show VA-Judger outperforms metric baselines in predicting human preferences on both in-domain and out-of-domain evaluations.
  arXiv  Detail & Related papers  (2026-08-19T06:53:14Z)
• Improving Text-to-Music Generation with Human Preference Rewards [9.458042125925573]
  We describe our entry to the efficiency track of the Academic Text-to-Music (ATTM) Grand Challenge at ICME 2026.<n>We add a learned human-preference reward from TuneJury, a twin pairwise ranker trained over open music-preference datasets.<n>The reward serves both as a training-time conditioning signal and as a sample-selection criterion.
  arXiv  Detail & Related papers  (2026-06-19T18:22:24Z)
• Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models [73.08789211016567]
  Diffusion LAIR is a reward-aware listwise preference optimization method for diffusion models.<n>Experiments show it outperforms strong preference optimization baselines on text-to-image generation, compositional generation, and image editing benchmarks.
  arXiv  Detail & Related papers  (2026-05-26T03:09:24Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Reference-Free Rating of LLM Responses via Latent Information [53.463883683503106]
  We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses.<n>We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals.<n>Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting.
  arXiv  Detail & Related papers  (2025-09-29T12:15:52Z)
• Calibrated Multi-Preference Optimization for Aligning Diffusion Models [90.15024547673785]
  Calibrated Preference Optimization (CaPO) is a novel method to align text-to-image (T2I) diffusion models.<n>CaPO incorporates the general preference from multiple reward models without human annotated data.<n> Experimental results show that CaPO consistently outperforms prior methods.
  arXiv  Detail & Related papers  (2025-02-04T18:59:23Z)
• Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts [17.243429150450886]
  We propose $textbfMulti-Preference Optimization (MPO) to optimize over entire sets of responses.<n>MPO employs deviation-based weighting, which emphasizes outlier responses that deviate most from the mean reward.<n>We theoretically prove that MPO reduces alignment bias at a rate of $mathcalOleft(frac1sqrtnright)$ with respect to the number of responses per query.
  arXiv  Detail & Related papers  (2024-12-05T21:50:22Z)
• Just Label the Repeats for In-The-Wild Audio-to-Score Alignment [7.7805314458791806]
  We propose an efficient workflow for alignment of in-the-wild performance audio and corresponding sheet music scans (images) We show that our proposed jump annotation workflow and improved feature representations together improve alignment accuracy by 150% relative to prior work.
  arXiv  Detail & Related papers  (2024-11-11T23:05:02Z)
• Joint Optimization of Ranking and Calibration with Contextualized Hybrid Model [24.66016187602343]
  We propose an approach that can Jointly optimize the Ranking and abilities (JRC) for short. JRC improves the ranking ability by contrasting the logit value for the sample with different labels and constrains the predicted probability to be a function of the logit subtraction. JRC has been deployed on the display advertising platform of Alibaba and has obtained significant performance improvements.
  arXiv  Detail & Related papers  (2022-08-12T08:32:13Z)
• Least Square Calibration for Peer Review [18.063450032460047]
  We propose a flexible framework, namely least square calibration (LSC), for selecting top candidates from peer ratings. Our framework provably performs perfect calibration from noiseless linear scoring functions under mild assumptions. Our algorithm consistently outperforms the baseline which select top papers based on the highest average ratings.
  arXiv  Detail & Related papers  (2021-10-25T02:40:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.