SongRewriter: A Chinese Song Rewriting System with Controllable Content and Rhyme Scheme

• URL: http://arxiv.org/abs/2211.15037v2
• Date: Fri, 26 May 2023 07:53:26 GMT
• Title: SongRewriter: A Chinese Song Rewriting System with Controllable Content and Rhyme Scheme
• Authors: Yusen Sun, Liangyou Li, Qun Liu and Dit-Yan Yeung
• Abstract summary: We propose a controllable Chinese lyrics generation and editing system which assists users without prior knowledge of melody composition. The system is trained by a randomized multi-level masking strategy which produces a unified model for generating entirely new lyrics or editing a few fragments.
• Score: 32.60994266892925
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Although lyrics generation has achieved significant progress in recent years, it has limited practical applications because the generated lyrics cannot be performed without composing compatible melodies. In this work, we bridge this practical gap by proposing a song rewriting system which rewrites the lyrics of an existing song such that the generated lyrics are compatible with the rhythm of the existing melody and thus singable. In particular, we propose SongRewriter,a controllable Chinese lyrics generation and editing system which assists users without prior knowledge of melody composition. The system is trained by a randomized multi-level masking strategy which produces a unified model for generating entirely new lyrics or editing a few fragments. To improve the controllabiliy of the generation process, we further incorporate a keyword prompt to control the lexical choices of the content and propose novel decoding constraints and a vowel modeling task to enable flexible end and internal rhyme schemes. While prior rhyming metrics are mainly for rap lyrics, we propose three novel rhyming evaluation metrics for song lyrics. Both automatic and human evaluations show that the proposed model performs better than the state-of-the-art models in both contents and rhyming quality.

Related papers

• Encoder-Decoder Framework for Interactive Free Verses with Generation with Controllable High-Quality Rhyming [15.595206559574017]
  We propose a novel fine-tuning approach that prepends the rhyming word at the start of each lyric. We conducted extensive experiments to compare this fine-tuning against the current state-of-the-art strategies for rhyming. We furnish a high-quality dataset in English and 12 other languages, analyse the approach's feasibility in a multilingual context, and propose metrics to compare methods in the future.
  arXiv  Detail & Related papers  (2024-05-08T16:13:40Z)
• SongComposer: A Large Language Model for Lyric and Melody Composition in Song Generation [88.33522730306674]
  SongComposer could understand and generate melodies and lyrics in symbolic song representations. We resort to symbolic song representation, the mature and efficient way humans designed for music. With extensive experiments, SongComposer demonstrates superior performance in lyric-to-melody generation, melody-to-lyric generation, song continuation, and text-to-song creation.
  arXiv  Detail & Related papers  (2024-02-27T16:15:28Z)
• Syllable-level lyrics generation from melody exploiting character-level language model [14.851295355381712]
  We propose to exploit fine-tuning character-level language models for syllable-level lyrics generation from symbolic melody. In particular, our method endeavors to incorporate linguistic knowledge of the language model into the beam search process of a syllable-level Transformer generator network.
  arXiv  Detail & Related papers  (2023-10-02T02:53:29Z)
• Unsupervised Melody-to-Lyric Generation [91.29447272400826]
  We propose a method for generating high-quality lyrics without training on any aligned melody-lyric data. We leverage the segmentation and rhythm alignment between melody and lyrics to compile the given melody into decoding constraints. Our model can generate high-quality lyrics that are more on-topic, singable, intelligible, and coherent than strong baselines.
  arXiv  Detail & Related papers  (2023-05-30T17:20:25Z)
• Unsupervised Melody-Guided Lyrics Generation [84.22469652275714]
  We propose to generate pleasantly listenable lyrics without training on melody-lyric aligned data. We leverage the crucial alignments between melody and lyrics and compile the given melody into constraints to guide the generation process.
  arXiv  Detail & Related papers  (2023-05-12T20:57:20Z)
• Deep Attention-Based Alignment Network for Melody Generation from Incomplete Lyrics [12.05359079565586]
  A deep neural lyrics-to-melody net is trained in an encoder-decoder way to predict possible pairs of lyrics-melody when given incomplete lyrics. The attention mechanism is exploited to align the predicted lyrics with the melody during the lyrics-to-melody generation.
  arXiv  Detail & Related papers  (2023-01-23T03:41:53Z)
• Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation [158.54649047794794]
  Re-creation of Creations (ROC) is a new paradigm for lyric-to-melody generation. ROC achieves good lyric-melody feature alignment in lyric-to-melody generation.
  arXiv  Detail & Related papers  (2022-08-11T08:44:47Z)
• Melody-Conditioned Lyrics Generation with SeqGANs [81.2302502902865]
  We propose an end-to-end melody-conditioned lyrics generation system based on Sequence Generative Adversarial Networks (SeqGAN) We show that the input conditions have no negative impact on the evaluation metrics while enabling the network to produce more meaningful results.
  arXiv  Detail & Related papers  (2020-10-28T02:35:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.