The pop song generator: designing an online course to teach collaborative, creative AI

• URL: http://arxiv.org/abs/2306.10069v1
• Date: Thu, 15 Jun 2023 18:17:28 GMT
• Title: The pop song generator: designing an online course to teach collaborative, creative AI
• Authors: Matthew Yee-king and Andrea Fiorucci and Mark d'Inverno
• Abstract summary: This article describes and evaluates a new online AI-creativity course. The course is based around three near-state-of-the-art AI models combined into a pop song generating system. A fine-tuned GPT-2 model writes lyrics, Music-VAE composes musical scores and instrumentation and Diffsinger synthesises a singing voice.
• Score: 1.2891210250935146
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This article describes and evaluates a new online AI-creativity course. The course is based around three near-state-of-the-art AI models combined into a pop song generating system. A fine-tuned GPT-2 model writes lyrics, Music-VAE composes musical scores and instrumentation and Diffsinger synthesises a singing voice. We explain the decisions made in designing the course which is based on Piagetian, constructivist 'learning-by-doing'. We present details of the five-week course design with learning objectives, technical concepts, and creative and technical activities. We explain how we overcame technical challenges to build a complete pop song generator system, consisting of Python scripts, pre-trained models, and Javascript code that runs in a dockerised Linux container via a web-based IDE. A quantitative analysis of student activity provides evidence on engagement and a benchmark for future improvements. A qualitative analysis of a workshop with experts validated the overall course design, it suggested the need for a stronger creative brief and ethical and legal content.

Related papers

• SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation [75.86473375730392]
  SongGen is a fully open-source, single-stage auto-regressive transformer for controllable song generation. It supports two output modes: mixed mode, which generates a mixture of vocals and accompaniment directly, and dual-track mode, which synthesizes them separately. To foster community engagement and future research, we will release our model weights, training code, annotated data, and preprocessing pipeline.
  arXiv  Detail & Related papers  (2025-02-18T18:52:21Z)
• Exploring the Collaborative Co-Creation Process with AI: A Case Study in Novice Music Production [3.3385152705660155]
  The study spanned the entire creative journey from ideation to releasing these songs on Spotify. Our findings highlight how AI transforms creative: accelerating ideation but compressing the traditional preparation stage. We propose the Human-AI Co-Creation Stage Model and the Human-AI Agency Model, offering new perspectives on collaborative co-creation with AI.
  arXiv  Detail & Related papers  (2025-01-25T17:00:17Z)
• AI TrackMate: Finally, Someone Who Will Give Your Music More Than Just "Sounds Great!" [4.886175454381699]
  Our framework integrates a Music Analysis Module, an LLM-Readable Music Report, and Music Production-Oriented Feedback Instruction. By bridging AI capabilities with the needs of independent producers, AI TrackMate offers on-demand analytical feedback. This system addresses the growing demand for objective self-assessment tools in the evolving landscape of independent music production.
  arXiv  Detail & Related papers  (2024-12-09T16:09:44Z)
• SongCreator: Lyrics-based Universal Song Generation [53.248473603201916]
  SongCreator is a song-generation system designed to tackle the challenge of generating songs with both vocals and accompaniment given lyrics. The model features two novel designs: a meticulously designed dual-sequence language model (M) to capture the information of vocals and accompaniment for song generation, and a series of attention mask strategies for DSLM. Experiments demonstrate the effectiveness of SongCreator by achieving state-of-the-art or competitive performances on all eight tasks.
  arXiv  Detail & Related papers  (2024-09-09T19:37:07Z)
• Foundation Models for Music: A Survey [77.77088584651268]
  Foundations models (FMs) have profoundly impacted diverse sectors, including music. This comprehensive review examines state-of-the-art (SOTA) pre-trained models and foundation models in music.
  arXiv  Detail & Related papers  (2024-08-26T15:13:14Z)
• Quantized GAN for Complex Music Generation from Dance Videos [48.196705493763986]
  We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates musical samples conditioned on dance videos. Our proposed framework takes dance video frames and human body motion as input, and learns to generate music samples that plausibly accompany the corresponding input.
  arXiv  Detail & Related papers  (2022-04-01T17:53:39Z)
• Introducing Variational Autoencoders to High School Students [12.341543369402217]
  This paper describes the lesson design and shares insights from the pilot studies with 22 students. We developed a web-based game and used Plato's cave, a philosophical metaphor, to introduce how VAEs work. We found that our approach was effective in teaching students about a novel AI concept.
  arXiv  Detail & Related papers  (2021-11-13T04:34:15Z)
• A Quantum Natural Language Processing Approach to Musical Intelligence [0.0]
  Quantum computing is a nascent technology, which is likely to impact the music industry in time to come. This work follows from previous experimental implementations of DisCoCat linguistic models on quantum hardware. We present Quanthoven, the first proof-of-concept ever built, which demonstrates that it is possible to program a quantum computer to learn to classify music.
  arXiv  Detail & Related papers  (2021-11-10T12:35:07Z)
• Music Composition with Deep Learning: A Review [1.7188280334580197]
  We analyze the ability of current Deep Learning models to generate music with creativity. We compare these models to the music composition process from a theoretical point of view.
  arXiv  Detail & Related papers  (2021-08-27T13:53:53Z)
• The MineRL BASALT Competition on Learning from Human Feedback [58.17897225617566]
  The MineRL BASALT competition aims to spur forward research on this important class of techniques. We design a suite of four tasks in Minecraft for which we expect it will be hard to write down hardcoded reward functions. We provide a dataset of human demonstrations on each of the four tasks, as well as an imitation learning baseline.
  arXiv  Detail & Related papers  (2021-07-05T12:18:17Z)
• RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning [69.20460466735852]
  This paper presents a deep reinforcement learning algorithm for online accompaniment generation. The proposed algorithm is able to respond to the human part and generate a melodic, harmonic and diverse machine part.
  arXiv  Detail & Related papers  (2020-02-08T03:53:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.