AI Song Contest: Human-AI Co-Creation in Songwriting

• URL: http://arxiv.org/abs/2010.05388v1
• Date: Mon, 12 Oct 2020 01:27:41 GMT
• Title: AI Song Contest: Human-AI Co-Creation in Songwriting
• Authors: Cheng-Zhi Anna Huang, Hendrik Vincent Koops, Ed Newton-Rex, Monica Dinculescu, Carrie J. Cai
• Abstract summary: We present findings on what 13 musician/developer teams, a total of 61 users, needed when co-creating a song with AI. We show how they leveraged and repurposed existing characteristics of AI to overcome some of these challenges. Findings reflect a need to design machine learning-powered music interfaces that are more decomposable, steerable, interpretable, and adaptive.
• Score: 8.399688944263843
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning is challenging the way we make music. Although research in deep generative models has dramatically improved the capability and fluency of music models, recent work has shown that it can be challenging for humans to partner with this new class of algorithms. In this paper, we present findings on what 13 musician/developer teams, a total of 61 users, needed when co-creating a song with AI, the challenges they faced, and how they leveraged and repurposed existing characteristics of AI to overcome some of these challenges. Many teams adopted modular approaches, such as independently running multiple smaller models that align with the musical building blocks of a song, before re-combining their results. As ML models are not easily steerable, teams also generated massive numbers of samples and curated them post-hoc, or used a range of strategies to direct the generation, or algorithmically ranked the samples. Ultimately, teams not only had to manage the "flare and focus" aspects of the creative process, but also juggle them with a parallel process of exploring and curating multiple ML models and outputs. These findings reflect a need to design machine learning-powered music interfaces that are more decomposable, steerable, interpretable, and adaptive, which in return will enable artists to more effectively explore how AI can extend their personal expression.

Related papers

• Prevailing Research Areas for Music AI in the Era of Foundation Models [8.067636023395236]
  There has been a surge of generative music AI applications within the past few years. We discuss the current state of music datasets and their limitations. We highlight applications of these generative models towards extensions to multiple modalities and integration with artists' workflow.
  arXiv  Detail & Related papers  (2024-09-14T09:06:43Z)
• SongCreator: Lyrics-based Universal Song Generation [53.248473603201916]
  SongCreator is a song-generation system designed to tackle the challenge of generating songs with both vocals and accompaniment given lyrics. The model features two novel designs: a meticulously designed dual-sequence language model (M) to capture the information of vocals and accompaniment for song generation, and a series of attention mask strategies for DSLM. Experiments demonstrate the effectiveness of SongCreator by achieving state-of-the-art or competitive performances on all eight tasks.
  arXiv  Detail & Related papers  (2024-09-09T19:37:07Z)
• Reducing Barriers to the Use of Marginalised Music Genres in AI [7.140590440016289]
  This project aims to explore the eXplainable AI (XAI) challenges and opportunities associated with reducing barriers to using marginalised genres of music with AI models. XAI opportunities identified included topics of improving transparency and control of AI models, explaining the ethics and bias of AI models, fine tuning large models with small datasets to reduce bias, and explaining style-transfer opportunities with AI models. We are now building on this project to bring together a global International Responsible AI Music community and invite people to join our network.
  arXiv  Detail & Related papers  (2024-07-18T12:10:04Z)
• MuPT: A Generative Symbolic Music Pretrained Transformer [56.09299510129221]
  We explore the application of Large Language Models (LLMs) to the pre-training of music. To address the challenges associated with misaligned measures from different tracks during generation, we propose a Synchronized Multi-Track ABC Notation (SMT-ABC Notation) Our contributions include a series of models capable of handling up to 8192 tokens, covering 90% of the symbolic music data in our training set.
  arXiv  Detail & Related papers  (2024-04-09T15:35:52Z)
• Exploring Variational Auto-Encoder Architectures, Configurations, and Datasets for Generative Music Explainable AI [7.391173255888337]
  Generative AI models for music and the arts are increasingly complex and hard to understand. One approach to making generative AI models more understandable is to impose a small number of semantically meaningful attributes on generative AI models. This paper contributes a systematic examination of the impact that different combinations of Variational Auto-Encoder models (MeasureVAE and AdversarialVAE) have on music generation performance.
  arXiv  Detail & Related papers  (2023-11-14T17:27:30Z)
• An Autoethnographic Exploration of XAI in Algorithmic Composition [7.775986202112564]
  This paper introduces an autoethnographic study of the use of the MeasureVAE generative music XAI model with interpretable latent dimensions trained on Irish music. Findings suggest that the exploratory nature of the music-making workflow foregrounds musical features of the training dataset rather than features of the generative model itself.
  arXiv  Detail & Related papers  (2023-08-11T12:03:17Z)
• Generating music with sentiment using Transformer-GANs [0.0]
  We propose a generative model of symbolic music conditioned by data retrieved from human sentiment. We try to tackle both of the problems above by employing an efficient linear version of Attention and using a Discriminator.
  arXiv  Detail & Related papers  (2022-12-21T15:59:35Z)
• WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models [91.92346150646007]
  In this work, we introduce WinoGAViL: an online game to collect vision-and-language associations. We use the game to collect 3.5K instances, finding that they are intuitive for humans but challenging for state-of-the-art AI models. Our analysis as well as the feedback we collect from players indicate that the collected associations require diverse reasoning skills.
  arXiv  Detail & Related papers  (2022-07-25T23:57:44Z)
• The MineRL BASALT Competition on Learning from Human Feedback [58.17897225617566]
  The MineRL BASALT competition aims to spur forward research on this important class of techniques. We design a suite of four tasks in Minecraft for which we expect it will be hard to write down hardcoded reward functions. We provide a dataset of human demonstrations on each of the four tasks, as well as an imitation learning baseline.
  arXiv  Detail & Related papers  (2021-07-05T12:18:17Z)
• Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learning [55.854205371307884]
  We formalize the music-conditioned dance generation as a sequence-to-sequence learning problem. We propose a novel curriculum learning strategy to alleviate error accumulation of autoregressive models in long motion sequence generation. Our approach significantly outperforms the existing state-of-the-arts on automatic metrics and human evaluation.
  arXiv  Detail & Related papers  (2020-06-11T00:08:25Z)
• RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning [69.20460466735852]
  This paper presents a deep reinforcement learning algorithm for online accompaniment generation. The proposed algorithm is able to respond to the human part and generate a melodic, harmonic and diverse machine part.
  arXiv  Detail & Related papers  (2020-02-08T03:53:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.