Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation
- URL: http://arxiv.org/abs/2208.05697v2
- Date: Fri, 12 Aug 2022 03:22:37 GMT
- Title: Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation
- Authors: Ang Lv, Xu Tan, Tao Qin, Tie-Yan Liu, Rui Yan
- Abstract summary: Re-creation of Creations (ROC) is a new paradigm for lyric-to-melody generation.
ROC achieves good lyric-melody feature alignment in lyric-to-melody generation.
- Score: 158.54649047794794
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Lyric-to-melody generation is an important task in songwriting, and is also
quite challenging due to its distinctive characteristics: the generated
melodies should not only follow good musical patterns, but also align with
features in lyrics such as rhythms and structures. These characteristics cannot
be well handled by neural generation models that learn lyric-to-melody mapping
in an end-to-end way, due to several issues: (1) lack of aligned lyric-melody
training data to sufficiently learn lyric-melody feature alignment; (2) lack of
controllability in generation to explicitly guarantee the lyric-melody feature
alignment. In this paper, we propose Re-creation of Creations (ROC), a new
paradigm for lyric-to-melody generation that addresses the above issues through
a generation-retrieval pipeline. Specifically, our paradigm has two stages: (1)
creation stage, where a huge amount of music pieces are generated by a
neural-based melody language model and indexed in a database through several
key features (e.g., chords, tonality, rhythm, and structural information
including chorus or verse); (2) re-creation stage, where melodies are recreated
by retrieving music pieces from the database according to the key features from
lyrics and concatenating best music pieces based on composition guidelines and
melody language model scores. Our new paradigm has several advantages: (1) It
only needs unpaired melody data to train melody language model, instead of
paired lyric-melody data in previous models. (2) It achieves good lyric-melody
feature alignment in lyric-to-melody generation. Experiments on English and
Chinese datasets demonstrate that ROC outperforms previous neural based
lyric-to-melody generation models on both objective and subjective metrics. We
provide our code in supplementary material and provide demos on github.
Related papers
- Controllable Lyrics-to-Melody Generation [14.15838552524433]
We propose a controllable lyrics-to-melody generation network, ConL2M, which is able to generate realistic melodies from lyrics in user-desired musical style.
Our work contains three main novelties: 1) To model the dependencies of music attributes cross multiple sequences, inter-branch memory fusion (Memofu) is proposed to enable information flow between multi-branch stacked LSTM architecture; 2) Reference style embedding (RSE) is proposed to improve the quality of generation as well as control the musical style of generated melodies; 3) Sequence-level statistical loss (SeqLoss) is proposed to help the model learn sequence-level
arXiv Detail & Related papers (2023-06-05T06:14:08Z) - Unsupervised Melody-to-Lyric Generation [91.29447272400826]
We propose a method for generating high-quality lyrics without training on any aligned melody-lyric data.
We leverage the segmentation and rhythm alignment between melody and lyrics to compile the given melody into decoding constraints.
Our model can generate high-quality lyrics that are more on-topic, singable, intelligible, and coherent than strong baselines.
arXiv Detail & Related papers (2023-05-30T17:20:25Z) - Unsupervised Melody-Guided Lyrics Generation [84.22469652275714]
We propose to generate pleasantly listenable lyrics without training on melody-lyric aligned data.
We leverage the crucial alignments between melody and lyrics and compile the given melody into constraints to guide the generation process.
arXiv Detail & Related papers (2023-05-12T20:57:20Z) - Interpretable Melody Generation from Lyrics with Discrete-Valued
Adversarial Training [12.02541352832997]
Gumbel-Softmax is exploited to solve the non-differentiability problem of generating music attributes by Generative Adversarial Networks (GANs)
Users can listen to the generated AI song as well as recreate a new song by selecting from recommended music attributes.
arXiv Detail & Related papers (2022-06-30T05:45:47Z) - TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage
Method [92.36505210982648]
TeleMelody is a two-stage lyric-to-melody generation system with music template.
It generates melodies with higher quality, better controllability, and less requirement on paired lyric-melody data.
arXiv Detail & Related papers (2021-09-20T15:19:33Z) - SongMASS: Automatic Song Writing with Pre-training and Alignment
Constraint [54.012194728496155]
SongMASS is proposed to overcome the challenges of lyric-to-melody generation and melody-to-lyric generation.
It leverages masked sequence to sequence (MASS) pre-training and attention based alignment modeling.
We show that SongMASS generates lyric and melody with significantly better quality than the baseline method.
arXiv Detail & Related papers (2020-12-09T16:56:59Z) - Melody-Conditioned Lyrics Generation with SeqGANs [81.2302502902865]
We propose an end-to-end melody-conditioned lyrics generation system based on Sequence Generative Adversarial Networks (SeqGAN)
We show that the input conditions have no negative impact on the evaluation metrics while enabling the network to produce more meaningful results.
arXiv Detail & Related papers (2020-10-28T02:35:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.