The pop song generator: designing an online course to teach collaborative, creative AI

• URL: http://arxiv.org/abs/2306.10069v1
• Date: Thu, 15 Jun 2023 18:17:28 GMT
• Title: The pop song generator: designing an online course to teach collaborative, creative AI
• Authors: Matthew Yee-king and Andrea Fiorucci and Mark d'Inverno
• Abstract summary: This article describes and evaluates a new online AI-creativity course. The course is based around three near-state-of-the-art AI models combined into a pop song generating system. A fine-tuned GPT-2 model writes lyrics, Music-VAE composes musical scores and instrumentation and Diffsinger synthesises a singing voice.
• Score: 1.2891210250935146
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This article describes and evaluates a new online AI-creativity course. The course is based around three near-state-of-the-art AI models combined into a pop song generating system. A fine-tuned GPT-2 model writes lyrics, Music-VAE composes musical scores and instrumentation and Diffsinger synthesises a singing voice. We explain the decisions made in designing the course which is based on Piagetian, constructivist 'learning-by-doing'. We present details of the five-week course design with learning objectives, technical concepts, and creative and technical activities. We explain how we overcame technical challenges to build a complete pop song generator system, consisting of Python scripts, pre-trained models, and Javascript code that runs in a dockerised Linux container via a web-based IDE. A quantitative analysis of student activity provides evidence on engagement and a benchmark for future improvements. A qualitative analysis of a workshop with experts validated the overall course design, it suggested the need for a stronger creative brief and ethical and legal content.

Related papers

• SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation [75.86473375730392]
  SongGen is a fully open-source, single-stage auto-regressive transformer for controllable song generation. It supports two output modes: mixed mode, which generates a mixture of vocals and accompaniment directly, and dual-track mode, which synthesizes them separately. To foster community engagement and future research, we will release our model weights, training code, annotated data, and preprocessing pipeline.
  arXiv  Detail & Related papers  (2025-02-18T18:52:21Z)
• Exploring the Collaborative Co-Creation Process with AI: A Case Study in Novice Music Production [3.3385152705660155]
  The study spanned the entire creative journey from ideation to releasing these songs on Spotify. Our findings highlight how AI transforms creative: accelerating ideation but compressing the traditional preparation stage. We propose the Human-AI Co-Creation Stage Model and the Human-AI Agency Model, offering new perspectives on collaborative co-creation with AI.
  arXiv  Detail & Related papers  (2025-01-25T17:00:17Z)
• AI TrackMate: Finally, Someone Who Will Give Your Music More Than Just "Sounds Great!" [4.886175454381699]
  Our framework integrates a Music Analysis Module, an LLM-Readable Music Report, and Music Production-Oriented Feedback Instruction. By bridging AI capabilities with the needs of independent producers, AI TrackMate offers on-demand analytical feedback. This system addresses the growing demand for objective self-assessment tools in the evolving landscape of independent music production.
  arXiv  Detail & Related papers  (2024-12-09T16:09:44Z)
• SongCreator: Lyrics-based Universal Song Generation [53.248473603201916]
  SongCreator is a song-generation system designed to tackle the challenge of generating songs with both vocals and accompaniment given lyrics. The model features two novel designs: a meticulously designed dual-sequence language model (M) to capture the information of vocals and accompaniment for song generation, and a series of attention mask strategies for DSLM. Experiments demonstrate the effectiveness of SongCreator by achieving state-of-the-art or competitive performances on all eight tasks.
  arXiv  Detail & Related papers  (2024-09-09T19:37:07Z)
• Foundation Models for Music: A Survey [77.77088584651268]
  Foundations models (FMs) have profoundly impacted diverse sectors, including music. This comprehensive review examines state-of-the-art (SOTA) pre-trained models and foundation models in music.
  arXiv  Detail & Related papers  (2024-08-26T15:13:14Z)
• PortfolioMentor: Multimodal Generative AI Companion for Learning and Crafting Interactive Digital Art Portfolios [1.8130068086063336]
  Digital art portfolios serve as impactful mediums for artists to convey their visions, weaving together visuals, audio, interactions, and narratives. Without technical backgrounds, design students often find it challenging to translate creative ideas into tangible codes and designs. This tool guides and collaborates with students through proactive suggestions and responsible Q&As for learning, inspiration, and support. In detail, the system starts with the understanding of the task and artist's visions, follows the co-creation of visual illustrations, audio or music suggestions and files, click-scroll effects for interactions, and creative vision conceptualization.
  arXiv  Detail & Related papers  (2023-11-23T16:36:40Z)
• Quantized GAN for Complex Music Generation from Dance Videos [48.196705493763986]
  We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates musical samples conditioned on dance videos. Our proposed framework takes dance video frames and human body motion as input, and learns to generate music samples that plausibly accompany the corresponding input.
  arXiv  Detail & Related papers  (2022-04-01T17:53:39Z)
• Introducing Variational Autoencoders to High School Students [12.341543369402217]
  This paper describes the lesson design and shares insights from the pilot studies with 22 students. We developed a web-based game and used Plato's cave, a philosophical metaphor, to introduce how VAEs work. We found that our approach was effective in teaching students about a novel AI concept.
  arXiv  Detail & Related papers  (2021-11-13T04:34:15Z)
• A Quantum Natural Language Processing Approach to Musical Intelligence [0.0]
  Quantum computing is a nascent technology, which is likely to impact the music industry in time to come. This work follows from previous experimental implementations of DisCoCat linguistic models on quantum hardware. We present Quanthoven, the first proof-of-concept ever built, which demonstrates that it is possible to program a quantum computer to learn to classify music.
  arXiv  Detail & Related papers  (2021-11-10T12:35:07Z)
• Music Composition with Deep Learning: A Review [1.7188280334580197]
  We analyze the ability of current Deep Learning models to generate music with creativity. We compare these models to the music composition process from a theoretical point of view.
  arXiv  Detail & Related papers  (2021-08-27T13:53:53Z)
• ProtoTransformer: A Meta-Learning Approach to Providing Student Feedback [54.142719510638614]
  In this paper, we frame the problem of providing feedback as few-shot classification. A meta-learner adapts to give feedback to student code on a new programming question from just a few examples by instructors. Our approach was successfully deployed to deliver feedback to 16,000 student exam-solutions in a programming course offered by a tier 1 university.
  arXiv  Detail & Related papers  (2021-07-23T22:41:28Z)
• The MineRL BASALT Competition on Learning from Human Feedback [58.17897225617566]
  The MineRL BASALT competition aims to spur forward research on this important class of techniques. We design a suite of four tasks in Minecraft for which we expect it will be hard to write down hardcoded reward functions. We provide a dataset of human demonstrations on each of the four tasks, as well as an imitation learning baseline.
  arXiv  Detail & Related papers  (2021-07-05T12:18:17Z)
• Can GAN originate new electronic dance music genres? -- Generating novel rhythm patterns using GAN with Genre Ambiguity Loss [0.0]
  This paper focuses on music generation, especially rhythm patterns of electronic dance music, and discusses if we can use deep learning to generate novel rhythms. We extend the framework of Generative Adversarial Networks(GAN) and encourage it to diverge from the dataset's inherent distributions. The paper shows that our proposed GAN can generate rhythm patterns that sound like music rhythms but do not belong to any genres in the training dataset.
  arXiv  Detail & Related papers  (2020-11-25T23:22:12Z)
• RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning [69.20460466735852]
  This paper presents a deep reinforcement learning algorithm for online accompaniment generation. The proposed algorithm is able to respond to the human part and generate a melodic, harmonic and diverse machine part.
  arXiv  Detail & Related papers  (2020-02-08T03:53:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.