A Training-Free Approach for Music Style Transfer with Latent Diffusion Models

• URL: http://arxiv.org/abs/2411.15913v1
• Date: Sun, 24 Nov 2024 16:53:34 GMT
• Title: A Training-Free Approach for Music Style Transfer with Latent Diffusion Models
• Authors: Sooyoung Kim, Joonwoo Kwon, Heehwan Wang, Shinjae Yoo, Yuewei Lin, Jiook Cha,
• Abstract summary: This paper introduces a novel training-free approach leveraging pre-trained Latent Diffusion Models (LDMs) By manipulating the self-attention features of the LDM, we effectively transfer the style of reference music onto content music without additional training.
• Score: 5.734429262507927
• License:
• Abstract: Music style transfer, while offering exciting possibilities for personalized music generation, often requires extensive training or detailed textual descriptions. This paper introduces a novel training-free approach leveraging pre-trained Latent Diffusion Models (LDMs). By manipulating the self-attention features of the LDM, we effectively transfer the style of reference music onto content music without additional training. Our method achieves superior style transfer and melody preservation compared to existing methods. This work opens new creative avenues for personalized music generation.

Related papers

• Foundation Models for Music: A Survey [77.77088584651268]
  Foundations models (FMs) have profoundly impacted diverse sectors, including music. This comprehensive review examines state-of-the-art (SOTA) pre-trained models and foundation models in music.
  arXiv  Detail & Related papers  (2024-08-26T15:13:14Z)
• MuPT: A Generative Symbolic Music Pretrained Transformer [56.09299510129221]
  We explore the application of Large Language Models (LLMs) to the pre-training of music. To address the challenges associated with misaligned measures from different tracks during generation, we propose a Synchronized Multi-Track ABC Notation (SMT-ABC Notation) Our contributions include a series of models capable of handling up to 8192 tokens, covering 90% of the symbolic music data in our training set.
  arXiv  Detail & Related papers  (2024-04-09T15:35:52Z)
• MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models [24.582948932985726]
  This paper introduces a novel approach to the editing of music generated by text-to-music models. Our method transforms text editing to textitlatent space manipulation while adding an extra constraint to enforce consistency. Experimental results demonstrate superior performance over both zero-shot and certain supervised baselines in style and timbre transfer evaluations.
  arXiv  Detail & Related papers  (2024-02-09T04:34:08Z)
• MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies [32.482588500419006]
  We build a state-of-the-art text-to-music model, MusicLDM, that adapts Stable Diffusion and AudioLDM architectures to the music domain. We propose two different mixup strategies for data augmentation: beat-synchronous audio mixup and beat-synchronous latent mixup. In addition to popular evaluation metrics, we design several new evaluation metrics based on CLAP score to demonstrate that our proposed MusicLDM and beat-synchronous mixup strategies improve both the quality and novelty of generated music.
  arXiv  Detail & Related papers  (2023-08-03T05:35:37Z)
• Transfer Learning for Underrepresented Music Generation [0.9645196221785693]
  We identify Iranian folk music as an example of such an OOD genre for MusicVAE, a large generative music model. We find that a combinational creativity transfer learning approach can efficiently adapt MusicVAE to an Iranian folk music dataset, indicating potential for generating underrepresented music genres in the future.
  arXiv  Detail & Related papers  (2023-06-01T01:53:10Z)
• GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework [58.64512825534638]
  Symbolic music generation aims to create musical notes, which can help users compose music. We introduce a framework known as GETMusic, with GET'' standing for GEnerate music Tracks'' GETScore represents musical notes as tokens and organizes tokens in a 2D structure, with tracks stacked vertically and progressing horizontally over time. Our proposed representation, coupled with the non-autoregressive generative model, empowers GETMusic to generate music with any arbitrary source-target track combinations.
  arXiv  Detail & Related papers  (2023-05-18T09:53:23Z)
• ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Models [67.66825818489406]
  This paper introduces a text-to-waveform music generation model, underpinned by the utilization of diffusion models. Our methodology hinges on the innovative incorporation of free-form textual prompts as conditional factors to guide the waveform generation process. We demonstrate that our generated music in the waveform domain outperforms previous works by a large margin in terms of diversity, quality, and text-music relevance.
  arXiv  Detail & Related papers  (2023-02-09T06:27:09Z)
• Personalized Popular Music Generation Using Imitation and Structure [1.971709238332434]
  We propose a statistical machine learning model that is able to capture and imitate the structure, melody, chord, and bass style from a given example seed song. An evaluation using 10 pop songs shows that our new representations and methods are able to create high-quality stylistic music.
  arXiv  Detail & Related papers  (2021-05-10T23:43:00Z)
• Self-Supervised VQ-VAE For One-Shot Music Style Transfer [2.6381163133447836]
  We present a novel method for one-shot timbre transfer based on an extension of the vector-quantized variational autoencoder (VQ-VAE) We evaluate the method using a set of objective metrics and show that it is able to outperform selected baselines.
  arXiv  Detail & Related papers  (2021-02-10T21:42:49Z)
• Unsupervised Cross-Domain Singing Voice Conversion [105.1021715879586]
  We present a wav-to-wav generative model for the task of singing voice conversion from any identity. Our method utilizes both an acoustic model, trained for the task of automatic speech recognition, together with melody extracted features to drive a waveform-based generator.
  arXiv  Detail & Related papers  (2020-08-06T18:29:11Z)
• Incorporating Music Knowledge in Continual Dataset Augmentation for Music Generation [69.06413031969674]
  Aug-Gen is a method of dataset augmentation for any music generation system trained on a resource-constrained domain. We apply Aug-Gen to Transformer-based chorale generation in the style of J.S. Bach, and show that this allows for longer training and results in better generative output.
  arXiv  Detail & Related papers  (2020-06-23T21:06:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.