Adaptive Accompaniment with ReaLchords

• URL: http://arxiv.org/abs/2506.14723v1
• Date: Tue, 17 Jun 2025 16:59:05 GMT
• Title: Adaptive Accompaniment with ReaLchords
• Authors: Yusong Wu, Tim Cooijmans, Kyle Kastner, Adam Roberts, Ian Simon, Alexander Scarlatos, Chris Donahue, Cassie Tarakajian, Shayegan Omidshafiei, Aaron Courville, Pablo Samuel Castro, Natasha Jaques, Cheng-Zhi Anna Huang,
• Abstract summary: We propose ReaLchords, an online generative model for improvising chord accompaniment to user melody.<n>We start with an online model pretrained by maximum likelihood, and use reinforcement learning to finetune the model for online use.
• Score: 60.690020661819055
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Jamming requires coordination, anticipation, and collaborative creativity between musicians. Current generative models of music produce expressive output but are not able to generate in an \emph{online} manner, meaning simultaneously with other musicians (human or otherwise). We propose ReaLchords, an online generative model for improvising chord accompaniment to user melody. We start with an online model pretrained by maximum likelihood, and use reinforcement learning to finetune the model for online use. The finetuning objective leverages both a novel reward model that provides feedback on both harmonic and temporal coherency between melody and chord, and a divergence term that implements a novel type of distillation from a teacher model that can see the future melody. Through quantitative experiments and listening tests, we demonstrate that the resulting model adapts well to unfamiliar input and produce fitting accompaniment. ReaLchords opens the door to live jamming, as well as simultaneous co-creation in other modalities.

Related papers

• MelodySim: Measuring Melody-aware Music Similarity for Plagiarism Detection [5.80717487254309]
  MelodySim is a melody-aware music similarity model and dataset for plagiarism detection.<n>By augmenting Slakh2100; an existing MIDI dataset, we generate variations of each piece while preserving the melody.<n>A user study confirms that positive pairs indeed contain similar melodies, with other musical tracks significantly changed.
  arXiv  Detail & Related papers  (2025-05-27T10:14:03Z)
• ImprovNet -- Generating Controllable Musical Improvisations with Iterative Corruption Refinement [6.873190001575463]
  ImprovNet is a transformer-based architecture that generates expressive and controllable musical improvisations.<n>It can perform cross-genre and intra-genre improvisations, harmonize melodies with genre-specific styles, and execute short prompt continuation and infilling tasks.
  arXiv  Detail & Related papers  (2025-02-06T21:45:38Z)
• MusicFlow: Cascaded Flow Matching for Text Guided Music Generation [53.63948108922333]
  MusicFlow is a cascaded text-to-music generation model based on flow matching. We leverage masked prediction as the training objective, enabling the model to generalize to other tasks such as music infilling and continuation.
  arXiv  Detail & Related papers  (2024-10-27T15:35:41Z)
• Hierarchical Generative Modeling of Melodic Vocal Contours in Hindustani Classical Music [3.491362957652171]
  We focus on generative modeling of singers' vocal melodies extracted from audio recordings. We propose GaMaDHaNi, a modular two-level hierarchy, consisting of a generative model on pitch contours, and a pitch contour to audio synthesis model.
  arXiv  Detail & Related papers  (2024-08-22T18:04:29Z)
• Unsupervised Melody-to-Lyric Generation [91.29447272400826]
  We propose a method for generating high-quality lyrics without training on any aligned melody-lyric data. We leverage the segmentation and rhythm alignment between melody and lyrics to compile the given melody into decoding constraints. Our model can generate high-quality lyrics that are more on-topic, singable, intelligible, and coherent than strong baselines.
  arXiv  Detail & Related papers  (2023-05-30T17:20:25Z)
• Unsupervised Melody-Guided Lyrics Generation [84.22469652275714]
  We propose to generate pleasantly listenable lyrics without training on melody-lyric aligned data. We leverage the crucial alignments between melody and lyrics and compile the given melody into constraints to guide the generation process.
  arXiv  Detail & Related papers  (2023-05-12T20:57:20Z)
• BacHMMachine: An Interpretable and Scalable Model for Algorithmic Harmonization for Four-part Baroque Chorales [23.64897650817862]
  BacHMMachine employs a "theory-driven" framework guided by music composition principles. It provides a probabilistic framework for learning key modulations and chordal progressions from a given melodic line. It results in vast decreases in computational burden and greater interpretability.
  arXiv  Detail & Related papers  (2021-09-15T23:39:45Z)
• Unsupervised Cross-Domain Singing Voice Conversion [105.1021715879586]
  We present a wav-to-wav generative model for the task of singing voice conversion from any identity. Our method utilizes both an acoustic model, trained for the task of automatic speech recognition, together with melody extracted features to drive a waveform-based generator.
  arXiv  Detail & Related papers  (2020-08-06T18:29:11Z)
• RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning [69.20460466735852]
  This paper presents a deep reinforcement learning algorithm for online accompaniment generation. The proposed algorithm is able to respond to the human part and generate a melodic, harmonic and diverse machine part.
  arXiv  Detail & Related papers  (2020-02-08T03:53:52Z)
• Continuous Melody Generation via Disentangled Short-Term Representations and Structural Conditions [14.786601824794369]
  We present a model for composing melodies given a user specified symbolic scenario combined with a previous music context. Our model is capable of generating long melodies by regarding 8-beat note sequences as basic units, and shares consistent rhythm pattern structure with another specific song. Results show that the music generated by our model tends to have salient repetition structures, rich motives, and stable rhythm patterns.
  arXiv  Detail & Related papers  (2020-02-05T06:23:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.