Musical Agent Systems: MACAT and MACataRT

• URL: http://arxiv.org/abs/2502.00023v1
• Date: Sun, 19 Jan 2025 22:04:09 GMT
• Title: Musical Agent Systems: MACAT and MACataRT
• Authors: Keon Ju M. Lee, Philippe Pasquier,
• Abstract summary: We introduce MACAT and MACataRT, two distinct musical agent systems crafted to enhance interactive music-making between human musicians and AI. MaCAT is optimized for agent-led performance, employing real-time synthesis and self-listening to shape its output autonomously. MacataRT provides a flexible environment for collaborative improvisation through audio mosaicing and sequence-based learning.
• Score: 6.349140286855134
• License:
• Abstract: Our research explores the development and application of musical agents, human-in-the-loop generative AI systems designed to support music performance and improvisation within co-creative spaces. We introduce MACAT and MACataRT, two distinct musical agent systems crafted to enhance interactive music-making between human musicians and AI. MACAT is optimized for agent-led performance, employing real-time synthesis and self-listening to shape its output autonomously, while MACataRT provides a flexible environment for collaborative improvisation through audio mosaicing and sequence-based learning. Both systems emphasize training on personalized, small datasets, fostering ethical and transparent AI engagement that respects artistic integrity. This research highlights how interactive, artist-centred generative AI can expand creative possibilities, empowering musicians to explore new forms of artistic expression in real-time, performance-driven and music improvisation contexts.

Related papers

• Exploring the Collaborative Co-Creation Process with AI: A Case Study in Novice Music Production [3.3385152705660155]
  The study spanned the entire creative journey from ideation to releasing these songs on Spotify. Our findings highlight how AI transforms creative: accelerating ideation but compressing the traditional preparation stage. We propose the Human-AI Co-Creation Stage Model and the Human-AI Agency Model, offering new perspectives on collaborative co-creation with AI.
  arXiv  Detail & Related papers  (2025-01-25T17:00:17Z)
• AI TrackMate: Finally, Someone Who Will Give Your Music More Than Just "Sounds Great!" [4.886175454381699]
  Our framework integrates a Music Analysis Module, an LLM-Readable Music Report, and Music Production-Oriented Feedback Instruction. By bridging AI capabilities with the needs of independent producers, AI TrackMate offers on-demand analytical feedback. This system addresses the growing demand for objective self-assessment tools in the evolving landscape of independent music production.
  arXiv  Detail & Related papers  (2024-12-09T16:09:44Z)
• A Survey of Foundation Models for Music Understanding [60.83532699497597]
  This work is one of the early reviews of the intersection of AI techniques and music understanding. We investigated, analyzed, and tested recent large-scale music foundation models in respect of their music comprehension abilities.
  arXiv  Detail & Related papers  (2024-09-15T03:34:14Z)
• Tidal MerzA: Combining affective modelling and autonomous code generation through Reinforcement Learning [3.6594988197536344]
  Tidal-MerzA is a system for collaborative performances between humans and a machine agent in the context of live coding. It fuses two foundational models: ALCAA (Affective Live Coding Autonomous Agent) and Tidal Fuzz, a computational framework.
  arXiv  Detail & Related papers  (2024-09-12T10:38:55Z)
• MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Models [57.47799823804519]
  We are inspired by how musicians compose music not just from a movie script, but also through visualizations. We propose MeLFusion, a model that can effectively use cues from a textual description and the corresponding image to synthesize music. Our exhaustive experimental evaluation suggests that adding visual information to the music synthesis pipeline significantly improves the quality of generated music.
  arXiv  Detail & Related papers  (2024-06-07T06:38:59Z)
• Designing and Evaluating Dialogue LLMs for Co-Creative Improvised Theatre [48.19823828240628]
  This study presents Large Language Models (LLMs) deployed in a month-long live show at the Edinburgh Festival Fringe. We explore the technical capabilities and constraints of on-the-spot multi-party dialogue. Our human-in-the-loop methodology underlines the challenges of these LLMs in generating context-relevant responses.
  arXiv  Detail & Related papers  (2024-05-11T23:19:42Z)
• ComposerX: Multi-Agent Symbolic Music Composition with LLMs [51.68908082829048]
  Music composition is a complex task that requires abilities to understand and generate information with long dependency and harmony constraints. Current LLMs easily fail in this task, generating ill-written music even when equipped with modern techniques like In-Context-Learning and Chain-of-Thoughts. We propose ComposerX, an agent-based symbolic music generation framework.
  arXiv  Detail & Related papers  (2024-04-28T06:17:42Z)
• Interactive Melody Generation System for Enhancing the Creativity of Musicians [0.0]
  This study proposes a system designed to enumerate the process of collaborative composition among humans. By integrating multiple Recurrent Neural Network (RNN) models, the system provides an experience akin to collaborating with several composers.
  arXiv  Detail & Related papers  (2024-03-06T01:33:48Z)
• MARBLE: Music Audio Representation Benchmark for Universal Evaluation [79.25065218663458]
  We introduce the Music Audio Representation Benchmark for universaL Evaluation, termed MARBLE. It aims to provide a benchmark for various Music Information Retrieval (MIR) tasks by defining a comprehensive taxonomy with four hierarchy levels, including acoustic, performance, score, and high-level description. We then establish a unified protocol based on 14 tasks on 8 public-available datasets, providing a fair and standard assessment of representations of all open-sourced pre-trained models developed on music recordings as baselines.
  arXiv  Detail & Related papers  (2023-06-18T12:56:46Z)
• Pathway to Future Symbiotic Creativity [76.20798455931603]
  We propose a classification of the creative system with a hierarchy of 5 classes, showing the pathway of creativity evolving from a mimic-human artist to a Machine artist in its own right. In art creation, it is necessary for machines to understand humans' mental states, including desires, appreciation, and emotions, humans also need to understand machines' creative capabilities and limitations. We propose a novel framework for building future Machine artists, which comes with the philosophy that a human-compatible AI system should be based on the "human-in-the-loop" principle.
  arXiv  Detail & Related papers  (2022-08-18T15:12:02Z)
• Flat latent manifolds for music improvisation between human and machine [9.571383193449648]
  We consider a music-generating algorithm as a counterpart to a human musician, in a setting where reciprocal improvisation is to lead to new experiences. In the learned model, we generate novel musical sequences by quantification in latent space. We provide empirical evidence for our method via a set of experiments on music and we deploy our model for an interactive jam session with a professional drummer.
  arXiv  Detail & Related papers  (2022-02-23T09:00:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.