LyricJam: A system for generating lyrics for live instrumental music
- URL: http://arxiv.org/abs/2106.01960v1
- Date: Thu, 3 Jun 2021 16:06:46 GMT
- Title: LyricJam: A system for generating lyrics for live instrumental music
- Authors: Olga Vechtomova, Gaurav Sahu, Dhruv Kumar
- Abstract summary: We describe a real-time system that receives a live audio stream from a jam session and generates lyric lines that are congruent with the live music being played.
Two novel approaches are proposed to align the learned latent spaces of audio and text representations.
- Score: 11.521519161773288
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We describe a real-time system that receives a live audio stream from a jam
session and generates lyric lines that are congruent with the live music being
played. Two novel approaches are proposed to align the learned latent spaces of
audio and text representations that allow the system to generate novel lyric
lines matching live instrumental music. One approach is based on adversarial
alignment of latent representations of audio and lyrics, while the other
approach learns to transfer the topology from the music latent space to the
lyric latent space. A user study with music artists using the system showed
that the system was useful not only in lyric composition, but also encouraged
the artists to improvise and find new musical expressions. Another user study
demonstrated that users preferred the lines generated using the proposed
methods to the lines generated by a baseline model.
Related papers
- A Real-Time Lyrics Alignment System Using Chroma And Phonetic Features
For Classical Vocal Performance [7.488651253072641]
The goal of real-time lyrics alignment is to take live singing audio as input and pinpoint the exact position within given lyrics on the fly.
The task can benefit real-world applications such as the automatic subtitling of live concerts or operas.
This paper presents a real-time lyrics alignment system for classical vocal performances with two contributions.
arXiv Detail & Related papers (2024-01-17T13:25:32Z) - Unsupervised Melody-to-Lyric Generation [91.29447272400826]
We propose a method for generating high-quality lyrics without training on any aligned melody-lyric data.
We leverage the segmentation and rhythm alignment between melody and lyrics to compile the given melody into decoding constraints.
Our model can generate high-quality lyrics that are more on-topic, singable, intelligible, and coherent than strong baselines.
arXiv Detail & Related papers (2023-05-30T17:20:25Z) - Unsupervised Melody-Guided Lyrics Generation [84.22469652275714]
We propose to generate pleasantly listenable lyrics without training on melody-lyric aligned data.
We leverage the crucial alignments between melody and lyrics and compile the given melody into constraints to guide the generation process.
arXiv Detail & Related papers (2023-05-12T20:57:20Z) - Noise2Music: Text-conditioned Music Generation with Diffusion Models [73.74580231353684]
We introduce Noise2Music, where a series of diffusion models is trained to generate high-quality 30-second music clips from text prompts.
We find that the generated audio is not only able to faithfully reflect key elements of the text prompt such as genre, tempo, instruments, mood, and era.
Pretrained large language models play a key role in this story -- they are used to generate paired text for the audio of the training set and to extract embeddings of the text prompts ingested by the diffusion models.
arXiv Detail & Related papers (2023-02-08T07:27:27Z) - SongRewriter: A Chinese Song Rewriting System with Controllable Content
and Rhyme Scheme [32.60994266892925]
We propose a controllable Chinese lyrics generation and editing system which assists users without prior knowledge of melody composition.
The system is trained by a randomized multi-level masking strategy which produces a unified model for generating entirely new lyrics or editing a few fragments.
arXiv Detail & Related papers (2022-11-28T03:52:05Z) - LyricJam Sonic: A Generative System for Real-Time Composition and
Musical Improvisation [13.269034230828032]
LyricJam Sonic is a novel tool for musicians to rediscover previous recordings, re-contextualize them with other recordings, and create original live music compositions in real-time.
A bi-modal AI-driven approach uses generated lyric lines to find matching audio clips from the artist's past studio recordings.
The intent is to keep the artists in a state of creative flow to music creation rather than taking them into an analytical/critical state of deliberately searching for past audio segments.
arXiv Detail & Related papers (2022-10-27T17:27:58Z) - Genre-conditioned Acoustic Models for Automatic Lyrics Transcription of
Polyphonic Music [73.73045854068384]
We propose to transcribe the lyrics of polyphonic music using a novel genre-conditioned network.
The proposed network adopts pre-trained model parameters, and incorporates the genre adapters between layers to capture different genre peculiarities for lyrics-genre pairs.
Our experiments show that the proposed genre-conditioned network outperforms the existing lyrics transcription systems.
arXiv Detail & Related papers (2022-04-07T09:15:46Z) - Youling: an AI-Assisted Lyrics Creation System [72.00418962906083]
This paper demonstrates textitYouling, an AI-assisted lyrics creation system, designed to collaborate with music creators.
In the lyrics generation process, textitYouling supports traditional one pass full-text generation mode as well as an interactive generation mode.
The system also provides a revision module which enables users to revise undesired sentences or words of lyrics repeatedly.
arXiv Detail & Related papers (2022-01-18T03:57:04Z) - Melody-Conditioned Lyrics Generation with SeqGANs [81.2302502902865]
We propose an end-to-end melody-conditioned lyrics generation system based on Sequence Generative Adversarial Networks (SeqGAN)
We show that the input conditions have no negative impact on the evaluation metrics while enabling the network to produce more meaningful results.
arXiv Detail & Related papers (2020-10-28T02:35:40Z) - Generation of lyrics lines conditioned on music audio clips [13.23722670386104]
A bimodal neural network model learns to generate lines conditioned on any given short audio clip.
The system is intended to serve as a creativity tool for songwriters.
arXiv Detail & Related papers (2020-09-30T01:22:58Z) - Shimon the Rapper: A Real-Time System for Human-Robot Interactive Rap
Battles [1.7403133838762446]
We present a system for real-time lyrical improvisation between a human and a robot in the style of hip hop.
Our system takes vocal input from a human rapper, analyzes the semantic meaning, and generates a response that is rapped back by a robot over a musical groove.
arXiv Detail & Related papers (2020-09-19T14:04:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.