Sing-On-Your-Beat: Simple Text-Controllable Accompaniment Generations

• URL: http://arxiv.org/abs/2411.01661v2
• Date: Tue, 12 Nov 2024 22:10:55 GMT
• Title: Sing-On-Your-Beat: Simple Text-Controllable Accompaniment Generations
• Authors: Quoc-Huy Trinh, Minh-Van Nguyen, Trong-Hieu Nguyen Mau, Khoa Tran, Thanh Do,
• Abstract summary: We propose a straightforward method that enables control over the accompaniment through text prompts. Through extensive experiments, we successfully generate 10-second accompaniments using vocal input and text control.
• Score: 5.56093728482997
• License:
• Abstract: Singing is one of the most cherished forms of human entertainment. However, creating a beautiful song requires an accompaniment that complements the vocals and aligns well with the song instruments and genre. With advancements in deep learning, previous research has focused on generating suitable accompaniments but often lacks precise alignment with the desired instrumentation and genre. To address this, we propose a straightforward method that enables control over the accompaniment through text prompts, allowing the generation of music that complements the vocals and aligns with the song instrumental and genre requirements. Through extensive experiments, we successfully generate 10-second accompaniments using vocal input and text control.

Related papers

• Song Form-aware Full-Song Text-to-Lyrics Generation with Multi-Level Granularity Syllable Count Control [13.702198736153582]
  We propose a framework for lyrics generation that enables multi-level syllable control at the word, phrase, line, and paragraph levels. Our approach generates complete lyrics conditioned on input text and song form, ensuring alignment with specified syllable constraints.
  arXiv  Detail & Related papers  (2024-11-20T07:57:58Z)
• SongCreator: Lyrics-based Universal Song Generation [53.248473603201916]
  SongCreator is a song-generation system designed to tackle the challenge of generating songs with both vocals and accompaniment given lyrics. The model features two novel designs: a meticulously designed dual-sequence language model (M) to capture the information of vocals and accompaniment for song generation, and a series of attention mask strategies for DSLM. Experiments demonstrate the effectiveness of SongCreator by achieving state-of-the-art or competitive performances on all eight tasks.
  arXiv  Detail & Related papers  (2024-09-09T19:37:07Z)
• REFFLY: Melody-Constrained Lyrics Editing Model [50.03960548399128]
  We introduce REFFLY, the first revision framework designed to edit arbitrary forms of plain text draft into high-quality, full-fledged song lyrics. Our approach ensures that the generated lyrics retain the original meaning of the draft, align with the melody, and adhere to the desired song structures.
  arXiv  Detail & Related papers  (2024-08-30T23:22:34Z)
• Accompanied Singing Voice Synthesis with Fully Text-controlled Melody [61.147446955297625]
  Text-to-song (TTSong) is a music generation task that synthesizes accompanied singing voices. We present MelodyLM, the first TTSong model that generates high-quality song pieces with fully text-controlled melodies.
  arXiv  Detail & Related papers  (2024-07-02T08:23:38Z)
• Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment [56.019288564115136]
  We propose a novel task called text-to-song synthesis which incorporating both vocals and accompaniments generation. We develop Melodist, a two-stage text-to-song method that consists of singing voice synthesis (SVS) and vocal-to-accompaniment (V2A) synthesis. evaluation results on our dataset demonstrate that Melodist can synthesize songs with comparable quality and style consistency.
  arXiv  Detail & Related papers  (2024-04-14T18:00:05Z)
• Unsupervised Melody-to-Lyric Generation [91.29447272400826]
  We propose a method for generating high-quality lyrics without training on any aligned melody-lyric data. We leverage the segmentation and rhythm alignment between melody and lyrics to compile the given melody into decoding constraints. Our model can generate high-quality lyrics that are more on-topic, singable, intelligible, and coherent than strong baselines.
  arXiv  Detail & Related papers  (2023-05-30T17:20:25Z)
• Unsupervised Melody-Guided Lyrics Generation [84.22469652275714]
  We propose to generate pleasantly listenable lyrics without training on melody-lyric aligned data. We leverage the crucial alignments between melody and lyrics and compile the given melody into constraints to guide the generation process.
  arXiv  Detail & Related papers  (2023-05-12T20:57:20Z)
• SingSong: Generating musical accompaniments from singing [35.819589427197464]
  We present SingSong, a system that generates instrumental music to accompany input vocals. In a pairwise comparison with the same vocal inputs, listeners expressed a significant preference for instrumentals generated by SingSong.
  arXiv  Detail & Related papers  (2023-01-30T04:53:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.