Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion

• URL: http://arxiv.org/abs/2402.14285v4
• Date: Wed, 25 Sep 2024 03:12:27 GMT
• Title: Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion
• Authors: Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli S Sastry, Siddharth Gururani, Sageev Oore, Yisong Yue,
• Abstract summary: Musical rules are often expressed in symbolic form on note characteristics, such as note density or chord progression. We propose Control Guidance (SCG), a novel guidance method that only requires forward evaluation of rule functions. SCG achieves training-free guidance for non-differentiable rules for the first time.
• Score: 32.961767438163676
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the problem of symbolic music generation (e.g., generating piano rolls), with a technical focus on non-differentiable rule guidance. Musical rules are often expressed in symbolic form on note characteristics, such as note density or chord progression, many of which are non-differentiable which pose a challenge when using them for guided diffusion. We propose Stochastic Control Guidance (SCG), a novel guidance method that only requires forward evaluation of rule functions that can work with pre-trained diffusion models in a plug-and-play way, thus achieving training-free guidance for non-differentiable rules for the first time. Additionally, we introduce a latent diffusion architecture for symbolic music generation with high time resolution, which can be composed with SCG in a plug-and-play fashion. Compared to standard strong baselines in symbolic music generation, this framework demonstrates marked advancements in music quality and rule-based controllability, outperforming current state-of-the-art generators in a variety of settings. For detailed demonstrations, code and model checkpoints, please visit our project website: https://scg-rule-guided-music.github.io/.

Related papers

• Scaling Self-Supervised Representation Learning for Symbolic Piano Performance [52.661197827466886]
  We study the capabilities of generative autoregressive transformer models trained on large amounts of symbolic solo-piano transcriptions.<n>We use a comparatively smaller, high-quality subset to finetune models to produce musical continuations, perform symbolic classification tasks, and produce general-purpose contrastive MIDI embeddings.
  arXiv  Detail & Related papers  (2025-06-30T14:00:14Z)
• Mamba-Diffusion Model with Learnable Wavelet for Controllable Symbolic Music Generation [5.083504224028769]
  We represent symbolic music as image-like pianorolls, facilitating the use of diffusion models for the generation of symbolic music.<n>This study introduces a novel diffusion model that incorporates our proposed Transformer-Mamba block and learnable wavelet transform.<n>Our evaluation shows that our method achieves compelling results in terms of music quality and controllability.
  arXiv  Detail & Related papers  (2025-05-06T08:44:52Z)
• An End-to-End Approach for Chord-Conditioned Song Generation [14.951089833579063]
  Song Generation task aims to synthesize music composed of vocals and accompaniment from given lyrics. To mitigate the issue, we introduce an important concept from music composition, namely chords to song generation networks. We propose a novel model termed Chord-Conditioned Song Generator (CSG) based on it.
  arXiv  Detail & Related papers  (2024-09-10T08:07:43Z)
• MMT-BERT: Chord-aware Symbolic Music Generation Based on Multitrack Music Transformer and MusicBERT [44.204383306879095]
  We propose a novel symbolic music representation and Generative Adversarial Network (GAN) framework specially designed for symbolic multitrack music generation. To build a robust multitrack music generator, we fine-tune a pre-trained MusicBERT model to serve as the discriminator, and incorporate relativistic standard loss.
  arXiv  Detail & Related papers  (2024-09-02T03:18:56Z)
• MuseBarControl: Enhancing Fine-Grained Control in Symbolic Music Generation through Pre-Training and Counterfactual Loss [51.85076222868963]
  We introduce a pre-training task designed to link control signals directly with corresponding musical tokens. We then implement a novel counterfactual loss that promotes better alignment between the generated music and the control prompts.
  arXiv  Detail & Related papers  (2024-07-05T08:08:22Z)
• MuPT: A Generative Symbolic Music Pretrained Transformer [56.09299510129221]
  We explore the application of Large Language Models (LLMs) to the pre-training of music. To address the challenges associated with misaligned measures from different tracks during generation, we propose a Synchronized Multi-Track ABC Notation (SMT-ABC Notation) Our contributions include a series of models capable of handling up to 8192 tokens, covering 90% of the symbolic music data in our training set.
  arXiv  Detail & Related papers  (2024-04-09T15:35:52Z)
• Simple and Controllable Music Generation [94.61958781346176]
  MusicGen is a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns.
  arXiv  Detail & Related papers  (2023-06-08T15:31:05Z)
• GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework [58.64512825534638]
  Symbolic music generation aims to create musical notes, which can help users compose music. We introduce a framework known as GETMusic, with GET'' standing for GEnerate music Tracks'' GETScore represents musical notes as tokens and organizes tokens in a 2D structure, with tracks stacked vertically and progressing horizontally over time. Our proposed representation, coupled with the non-autoregressive generative model, empowers GETMusic to generate music with any arbitrary source-target track combinations.
  arXiv  Detail & Related papers  (2023-05-18T09:53:23Z)
• Universal Guidance for Diffusion Models [54.99356512898613]
  We propose a universal guidance algorithm that enables diffusion models to be controlled by arbitrary guidance modalities without the need to retrain any use-specific components. We show that our algorithm successfully generates quality images with guidance functions including segmentation, face recognition, object detection, and classifier signals.
  arXiv  Detail & Related papers  (2023-02-14T15:30:44Z)
• FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control [25.95359681751144]
  We propose the self-supervised description-to-sequence task, which allows for fine-grained controllable generation on a global level. We do so by extracting high-level features about the target sequence and learning the conditional distribution of sequences given the corresponding high-level description in a sequence-to-sequence modelling setup. By combining learned high level features with domain knowledge, which acts as a strong inductive bias, the model achieves state-of-the-art results in controllable symbolic music generation and generalizes well beyond the training distribution.
  arXiv  Detail & Related papers  (2022-01-26T13:51:19Z)
• Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework [3.029434408969759]
  We present a novel approach for calculating the positivity or negativity of a chord progression within a lead sheet. Our approach is similar to a Neural Machine Translation (NMT) problem, as we include high-level conditions in the encoder part of the sequence-to-sequence architectures. The proposed strategy is able to generate lead sheets in a controllable manner, resulting in distributions of musical attributes similar to those of the training dataset.
  arXiv  Detail & Related papers  (2021-04-27T09:04:21Z)
• Continuous Melody Generation via Disentangled Short-Term Representations and Structural Conditions [14.786601824794369]
  We present a model for composing melodies given a user specified symbolic scenario combined with a previous music context. Our model is capable of generating long melodies by regarding 8-beat note sequences as basic units, and shares consistent rhythm pattern structure with another specific song. Results show that the music generated by our model tends to have salient repetition structures, rich motives, and stable rhythm patterns.
  arXiv  Detail & Related papers  (2020-02-05T06:23:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.