Prevailing Research Areas for Music AI in the Era of Foundation Models

• URL: http://arxiv.org/abs/2409.09378v1
• Date: Sat, 14 Sep 2024 09:06:43 GMT
• Title: Prevailing Research Areas for Music AI in the Era of Foundation Models
• Authors: Megan Wei, Mateusz Modrzejewski, Aswin Sivaraman, Dorien Herremans,
• Abstract summary: There has been a surge of generative music AI applications within the past few years. We discuss the current state of music datasets and their limitations. We highlight applications of these generative models towards extensions to multiple modalities and integration with artists' workflow.
• Score: 8.067636023395236
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In tandem with the recent advancements in foundation model research, there has been a surge of generative music AI applications within the past few years. As the idea of AI-generated or AI-augmented music becomes more mainstream, many researchers in the music AI community may be wondering what avenues of research are left. With regards to music generative models, we outline the current areas of research with significant room for exploration. Firstly, we pose the question of foundational representation of these generative models and investigate approaches towards explainability. Next, we discuss the current state of music datasets and their limitations. We then overview different generative models, forms of evaluating these models, and their computational constraints/limitations. Subsequently, we highlight applications of these generative models towards extensions to multiple modalities and integration with artists' workflow as well as music education systems. Finally, we survey the potential copyright implications of generative music and discuss strategies for protecting the rights of musicians. While it is not meant to be exhaustive, our survey calls to attention a variety of research directions enabled by music foundation models.

Related papers

• A Survey of Foundation Models for Music Understanding [60.83532699497597]
  This work is one of the early reviews of the intersection of AI techniques and music understanding. We investigated, analyzed, and tested recent large-scale music foundation models in respect of their music comprehension abilities.
  arXiv  Detail & Related papers  (2024-09-15T03:34:14Z)
• Foundation Models for Music: A Survey [77.77088584651268]
  Foundations models (FMs) have profoundly impacted diverse sectors, including music. This comprehensive review examines state-of-the-art (SOTA) pre-trained models and foundation models in music.
  arXiv  Detail & Related papers  (2024-08-26T15:13:14Z)
• Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations [52.11801730860999]
  In recent years, the robot learning community has shown increasing interest in using deep generative models to capture the complexity of large datasets. We present the different types of models that the community has explored, such as energy-based models, diffusion models, action value maps, or generative adversarial networks. We also present the different types of applications in which deep generative models have been used, from grasp generation to trajectory generation or cost learning.
  arXiv  Detail & Related papers  (2024-08-08T11:34:31Z)
• Reducing Barriers to the Use of Marginalised Music Genres in AI [7.140590440016289]
  This project aims to explore the eXplainable AI (XAI) challenges and opportunities associated with reducing barriers to using marginalised genres of music with AI models. XAI opportunities identified included topics of improving transparency and control of AI models, explaining the ethics and bias of AI models, fine tuning large models with small datasets to reduce bias, and explaining style-transfer opportunities with AI models. We are now building on this project to bring together a global International Responsible AI Music community and invite people to join our network.
  arXiv  Detail & Related papers  (2024-07-18T12:10:04Z)
• MuPT: A Generative Symbolic Music Pretrained Transformer [56.09299510129221]
  We explore the application of Large Language Models (LLMs) to the pre-training of music. To address the challenges associated with misaligned measures from different tracks during generation, we propose a Synchronized Multi-Track ABC Notation (SMT-ABC Notation) Our contributions include a series of models capable of handling up to 8192 tokens, covering 90% of the symbolic music data in our training set.
  arXiv  Detail & Related papers  (2024-04-09T15:35:52Z)
• Exploring Variational Auto-Encoder Architectures, Configurations, and Datasets for Generative Music Explainable AI [7.391173255888337]
  Generative AI models for music and the arts are increasingly complex and hard to understand. One approach to making generative AI models more understandable is to impose a small number of semantically meaningful attributes on generative AI models. This paper contributes a systematic examination of the impact that different combinations of Variational Auto-Encoder models (MeasureVAE and AdversarialVAE) have on music generation performance.
  arXiv  Detail & Related papers  (2023-11-14T17:27:30Z)
• An Autoethnographic Exploration of XAI in Algorithmic Composition [7.775986202112564]
  This paper introduces an autoethnographic study of the use of the MeasureVAE generative music XAI model with interpretable latent dimensions trained on Irish music. Findings suggest that the exploratory nature of the music-making workflow foregrounds musical features of the training dataset rather than features of the generative model itself.
  arXiv  Detail & Related papers  (2023-08-11T12:03:17Z)
• MARBLE: Music Audio Representation Benchmark for Universal Evaluation [79.25065218663458]
  We introduce the Music Audio Representation Benchmark for universaL Evaluation, termed MARBLE. It aims to provide a benchmark for various Music Information Retrieval (MIR) tasks by defining a comprehensive taxonomy with four hierarchy levels, including acoustic, performance, score, and high-level description. We then establish a unified protocol based on 14 tasks on 8 public-available datasets, providing a fair and standard assessment of representations of all open-sourced pre-trained models developed on music recordings as baselines.
  arXiv  Detail & Related papers  (2023-06-18T12:56:46Z)
• A Survey on Artificial Intelligence for Music Generation: Agents, Domains and Perspectives [10.349825060515181]
  We describe how humans compose music and how new AI systems could imitate such process. To understand how AI models and algorithms generate music, we explore, analyze and describe the agents that take part of the music generation process.
  arXiv  Detail & Related papers  (2022-10-25T11:54:30Z)
• Artificial Musical Intelligence: A Survey [51.477064918121336]
  Music has become an increasingly prevalent domain of machine learning and artificial intelligence research. This article provides a definition of musical intelligence, introduces a taxonomy of its constituent components, and surveys the wide range of AI methods that can be, and have been, brought to bear in its pursuit.
  arXiv  Detail & Related papers  (2020-06-17T04:46:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.