Making the Most of Limited Data: Score-Aware Training for Text-to-Music Generation

• URL: http://arxiv.org/abs/2606.07387v1
• Date: Fri, 05 Jun 2026 15:24:36 GMT
• Title: Making the Most of Limited Data: Score-Aware Training for Text-to-Music Generation
• Abstract summary: State-of-the-art text-to-music generation systems rely on massive proprietary datasets and industrial-scale compute.<n>We propose textitscore-aware training, which treats audio-caption alignment score as a direct supervision signal throughout the pipeline.
• Score: 1.0323063834827415
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art text-to-music generation systems rely on massive proprietary datasets and industrial-scale compute, making it impossible to disentangle architectural contributions from resource advantages. We propose \textit{score-aware training}, which treats audio-caption alignment score as a direct supervision signal throughout the pipeline. Rather than discarding low-scoring segments, we repurpose them via a CLAP-conditioned Beta noise timestep schedule that routes them to high-noise training regimes, acting as an effective implicit regularizer. Complementarily, segment-level filtering removes the most misaligned examples, and a two-stage caption procedure bridges the distribution gap between verbose training captions and concise inference prompts. A REPA auxiliary loss further transfers structured semantic knowledge from pretrained CLAP and MuQ encoders without additional data. Our 450M-parameter FluxAudio-based system, submitted to the ICME 2026 ATTM Grand Challenge Efficiency Track, ranked 2nd across both tracks in the objective evaluation and 3rd in the Efficiency Track in the final MOS evaluation.

Related papers

• TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models [58.80225070159013]
  Large audio-language models (LALMs) describe audio at the clip level but cannot assign timestamps to the events, speakers, or sounds they identify.<n>We present TEMPO, the first unified model to handle audio, speech, and music timestamping tasks.
  arXiv  Detail & Related papers  (2026-08-30T19:49:21Z)
• SPECTRA: Subspace-Preserving Embedding Calibration, Transport, and Replay for Fully Few-Shot Class-Incremental Audio Classification [10.046325615622164]
  We propose SPECTRA, a framework built on a frozen encoder which adds three components.<n>On three FFCAC benchmarks, SPECTRA improves average accuracy and reduces forgetting over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-08-25T18:43:58Z)
• Listen, Reason, and Segment: Aligning LALMs with Editorial Judgment for Media Chapterization [58.34381853842135]
  Large Audio Language Models (LALMs) have made rapid progress on standardized benchmarks, yet their deployment in practical media, curation, archival indexing, and content distribution remains largely unrealized.<n>We identify automated audio chapterization, the task of segmenting continuous audio streams into thematically coherent chapters.<n>We present AudioChaps, a framework for aligning end-to-end LALMs for this task.
  arXiv  Detail & Related papers  (2026-08-17T13:14:01Z)
• Improving Text-to-Music Generation with Human Preference Rewards [9.458042125925573]
  We describe our entry to the efficiency track of the Academic Text-to-Music (ATTM) Grand Challenge at ICME 2026.<n>We add a learned human-preference reward from TuneJury, a twin pairwise ranker trained over open music-preference datasets.<n>The reward serves both as a training-time conditioning signal and as a sample-selection criterion.
  arXiv  Detail & Related papers  (2026-06-19T18:22:24Z)
• AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following [17.52339026461815]
  We introduce a novel dynamic rubric-based evaluation paradigm that adaptively decomposes complex audio captions into a variable number of independent, verifiable binary rubric items.<n>We benchmark this capability with the AnyAudio-Judge Bench, a comprehensive, bilingual benchmark comprising 7,920 meticulously curated samples.<n>Experiments demonstrate that AnyAudio-Judge not only significantly enhances zero-shot alignment detection compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-06-02T04:00:32Z)
• Robust Self-Training with Closed-loop Label Correction for Learning from Noisy Labels [29.68163073890278]
  Training deep neural networks with noisy labels remains a significant challenge.<n>We propose a self-training label correction framework using decoupled bilevel optimization.<n>Our method employs noisy posterior simulation and intermediate features to transfer ground-truth knowledge.
  arXiv  Detail & Related papers  (2026-03-14T11:10:17Z)
• Representation-Regularized Convolutional Audio Transformer for Audio Understanding [53.092757178419355]
  bootstrapping representations from scratch is computationally expensive, often requiring extensive training to converge.<n>We propose the Convolutional Audio Transformer (CAT), a unified framework designed to address these challenges.
  arXiv  Detail & Related papers  (2026-01-29T12:16:19Z)
• Pretraining End-to-End Keyword Search with Automatically Discovered Acoustic Units [8.86336076082867]
  We propose a method for pretraining E2E KWS systems with untranscribed data. We show that finetuning such a model significantly outperforms a model trained from scratch.
  arXiv  Detail & Related papers  (2024-07-05T17:07:58Z)
• End-to-End Real-World Polyphonic Piano Audio-to-Score Transcription with Hierarchical Decoding [4.604877755214193]
  Existing end-to-end piano A2S systems have been trained and evaluated with only synthetic data. We propose a sequence-to-sequence (Seq2Seq) model with a hierarchical decoder that aligns with the hierarchical structure of musical scores. We propose a two-stage training scheme, which involves pre-training the model using an expressive performance rendering system on synthetic audio, followed by fine-tuning the model using recordings of human performance.
  arXiv  Detail & Related papers  (2024-05-22T10:52:04Z)
• Aligned Unsupervised Pretraining of Object Detectors with Self-training [41.03780087924593]
  Unsupervised pretraining of object detectors has recently become a key component of object detector training. We propose a framework that mitigates this issue and consists of three simple yet key ingredients. We show that our strategy is also capable of pretraining from scratch (including the backbone) and works on complex images like COCO.
  arXiv  Detail & Related papers  (2023-07-28T17:46:00Z)
• Efficient Audio Captioning Transformer with Patchout and Text Guidance [74.59739661383726]
  We propose a full Transformer architecture that utilizes Patchout as proposed in [1], significantly reducing the computational complexity and avoiding overfitting. The caption generation is partly conditioned on textual AudioSet tags extracted by a pre-trained classification model. Our proposed method received the Judges Award at the Task6A of DCASE Challenge 2022.
  arXiv  Detail & Related papers  (2023-04-06T07:58:27Z)
• Audio-Visual Efficient Conformer for Robust Speech Recognition [91.3755431537592]
  We propose to improve the noise of the recently proposed Efficient Conformer Connectionist Temporal Classification architecture by processing both audio and visual modalities. Our experiments show that using audio and visual modalities allows to better recognize speech in the presence of environmental noise and significantly accelerate training, reaching lower WER with 4 times less training steps.
  arXiv  Detail & Related papers  (2023-01-04T05:36:56Z)
• Capturing scattered discriminative information using a deep architecture in acoustic scene classification [49.86640645460706]
  In this study, we investigate various methods to capture discriminative information and simultaneously mitigate the overfitting problem. We adopt a max feature map method to replace conventional non-linear activations in a deep neural network. Two data augment methods and two deep architecture modules are further explored to reduce overfitting and sustain the system's discriminative power.
  arXiv  Detail & Related papers  (2020-07-09T08:32:06Z)
• Deep Speaker Embeddings for Far-Field Speaker Recognition on Short Utterances [53.063441357826484]
  Speaker recognition systems based on deep speaker embeddings have achieved significant performance in controlled conditions. Speaker verification on short utterances in uncontrolled noisy environment conditions is one of the most challenging and highly demanded tasks. This paper presents approaches aimed to achieve two goals: a) improve the quality of far-field speaker verification systems in the presence of environmental noise, reverberation and b) reduce the system qualitydegradation for short utterances.
  arXiv  Detail & Related papers  (2020-02-14T13:34:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.