REFFLY: Melody-Constrained Lyrics Editing Model

• URL: http://arxiv.org/abs/2409.00292v1
• Date: Fri, 30 Aug 2024 23:22:34 GMT
• Title: REFFLY: Melody-Constrained Lyrics Editing Model
• Authors: Songyan Zhao, Bingxuan Li, Yufei Tian, Nanyun Peng,
• Abstract summary: We introduce REFFLY, the first revision framework designed to edit arbitrary forms of plain text draft into high-quality, full-fledged song lyrics. Our approach ensures that the generated lyrics retain the original meaning of the draft, align with the melody, and adhere to the desired song structures.
• Score: 50.03960548399128
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic melody-to-lyric generation aims to produce lyrics that align with a given melody. Although previous work can generate lyrics based on high-level control signals, such as keywords or genre, they often struggle with three challenges: (1) lack of controllability, as prior works are only able to produce lyrics from scratch, with little or no control over the content; (2) inability to generate fully structured songs with the desired format; and (3) failure to align prominent words in the lyrics with prominent notes in the melody, resulting in poor lyrics-melody alignment. In this work, we introduce REFFLY (REvision Framework For Lyrics), the first revision framework designed to edit arbitrary forms of plain text draft into high-quality, full-fledged song lyrics. Our approach ensures that the generated lyrics retain the original meaning of the draft, align with the melody, and adhere to the desired song structures. We demonstrate that REFFLY performs well in diverse task settings, such as lyrics revision and song translation. Experimental results show that our model outperforms strong baselines, such as Lyra (Tian et al. 2023) and GPT-4, by 25% in both musicality and text quality.

Related papers

• Song Form-aware Full-Song Text-to-Lyrics Generation with Multi-Level Granularity Syllable Count Control [13.702198736153582]
  We propose a framework for lyrics generation that enables multi-level syllable control at the word, phrase, line, and paragraph levels. Our approach generates complete lyrics conditioned on input text and song form, ensuring alignment with specified syllable constraints.
  arXiv  Detail & Related papers  (2024-11-20T07:57:58Z)
• Unsupervised Melody-to-Lyric Generation [91.29447272400826]
  We propose a method for generating high-quality lyrics without training on any aligned melody-lyric data. We leverage the segmentation and rhythm alignment between melody and lyrics to compile the given melody into decoding constraints. Our model can generate high-quality lyrics that are more on-topic, singable, intelligible, and coherent than strong baselines.
  arXiv  Detail & Related papers  (2023-05-30T17:20:25Z)
• Unsupervised Melody-Guided Lyrics Generation [84.22469652275714]
  We propose to generate pleasantly listenable lyrics without training on melody-lyric aligned data. We leverage the crucial alignments between melody and lyrics and compile the given melody into constraints to guide the generation process.
  arXiv  Detail & Related papers  (2023-05-12T20:57:20Z)
• Deep Attention-Based Alignment Network for Melody Generation from Incomplete Lyrics [12.05359079565586]
  A deep neural lyrics-to-melody net is trained in an encoder-decoder way to predict possible pairs of lyrics-melody when given incomplete lyrics. The attention mechanism is exploited to align the predicted lyrics with the melody during the lyrics-to-melody generation.
  arXiv  Detail & Related papers  (2023-01-23T03:41:53Z)
• SongRewriter: A Chinese Song Rewriting System with Controllable Content and Rhyme Scheme [32.60994266892925]
  We propose a controllable Chinese lyrics generation and editing system which assists users without prior knowledge of melody composition. The system is trained by a randomized multi-level masking strategy which produces a unified model for generating entirely new lyrics or editing a few fragments.
  arXiv  Detail & Related papers  (2022-11-28T03:52:05Z)
• Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation [158.54649047794794]
  Re-creation of Creations (ROC) is a new paradigm for lyric-to-melody generation. ROC achieves good lyric-melody feature alignment in lyric-to-melody generation.
  arXiv  Detail & Related papers  (2022-08-11T08:44:47Z)
• TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method [92.36505210982648]
  TeleMelody is a two-stage lyric-to-melody generation system with music template. It generates melodies with higher quality, better controllability, and less requirement on paired lyric-melody data.
  arXiv  Detail & Related papers  (2021-09-20T15:19:33Z)
• Melody-Conditioned Lyrics Generation with SeqGANs [81.2302502902865]
  We propose an end-to-end melody-conditioned lyrics generation system based on Sequence Generative Adversarial Networks (SeqGAN) We show that the input conditions have no negative impact on the evaluation metrics while enabling the network to produce more meaningful results.
  arXiv  Detail & Related papers  (2020-10-28T02:35:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.