Towards Assessing Data Replication in Music Generation with Music Similarity Metrics on Raw Audio

• URL: http://arxiv.org/abs/2407.14364v2
• Date: Thu, 1 Aug 2024 11:16:30 GMT
• Title: Towards Assessing Data Replication in Music Generation with Music Similarity Metrics on Raw Audio
• Authors: Roser Batlle-Roca, Wei-Hisang Liao, Xavier Serra, Yuki Mitsufuji, Emilia Gómez,
• Abstract summary: We present a model-independent open evaluation method based on diverse audio music similarity metrics to assess data replication. Our results show that the proposed methodology can estimate exact data replication with a proportion higher than 10%.
• Score: 25.254669525489923
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in music generation are raising multiple concerns about the implications of AI in creative music processes, current business models and impacts related to intellectual property management. A relevant discussion and related technical challenge is the potential replication and plagiarism of the training set in AI-generated music, which could lead to misuse of data and intellectual property rights violations. To tackle this issue, we present the Music Replication Assessment (MiRA) tool: a model-independent open evaluation method based on diverse audio music similarity metrics to assess data replication. We evaluate the ability of five metrics to identify exact replication by conducting a controlled replication experiment in different music genres using synthetic samples. Our results show that the proposed methodology can estimate exact data replication with a proportion higher than 10%. By introducing the MiRA tool, we intend to encourage the open evaluation of music-generative models by researchers, developers, and users concerning data replication, highlighting the importance of the ethical, social, legal, and economic consequences. Code and examples are available for reproducibility purposes.

Related papers

• Between the AI and Me: Analysing Listeners' Perspectives on AI- and Human-Composed Progressive Metal Music [1.2874569408514918]
  We explore participants' perspectives on AI- vs human-generated progressive metal, using rock music as a control group. We propose a mixed methods approach to assess the effects of generation type (human vs. AI), genre (progressive metal vs. rock), and curation process (random vs. cherry-picked) Our findings validate the use of fine-tuning to achieve genre-specific specialization in AI music generation. Despite some AI-generated excerpts receiving similar ratings to human music, listeners exhibited a preference for human compositions.
  arXiv  Detail & Related papers  (2024-07-31T14:03:45Z)
• Analyzing and reducing the synthetic-to-real transfer gap in Music Information Retrieval: the task of automatic drum transcription [0.6554326244334866]
  A popular method used to increase the amount of data is by generating them synthetically from music scores rendered with virtual instruments. This method can produce a virtually infinite quantity of tracks, but empirical evidence shows that models trained on previously created synthetic datasets do not transfer well to real tracks. In this work, besides increasing the amount of data, we identify and evaluate three more strategies that practitioners can use to improve the realism of the generated data.
  arXiv  Detail & Related papers  (2024-07-29T09:17:16Z)
• MuPT: A Generative Symbolic Music Pretrained Transformer [56.09299510129221]
  We explore the application of Large Language Models (LLMs) to the pre-training of music. To address the challenges associated with misaligned measures from different tracks during generation, we propose a Synchronized Multi-Track ABC Notation (SMT-ABC Notation) Our contributions include a series of models capable of handling up to 8192 tokens, covering 90% of the symbolic music data in our training set.
  arXiv  Detail & Related papers  (2024-04-09T15:35:52Z)
• A Survey of Music Generation in the Context of Interaction [3.6522809408725223]
  Machine learning has been successfully used to compose and generate music, both melodies and polyphonic pieces. Most of these models are not suitable for human-machine co-creation through live interaction.
  arXiv  Detail & Related papers  (2024-02-23T12:41:44Z)
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection [74.51523859064802]
  We introduce a new framework called Self-Reflective Retrieval-Augmented Generation (Self-RAG) Self-RAG enhances an LM's quality and factuality through retrieval and self-reflection. It significantly outperforms state-of-the-art LLMs and retrieval-augmented models on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-10-17T18:18:32Z)
• A Comprehensive Survey for Evaluation Methodologies of AI-Generated Music [14.453416870193072]
  This study aims to comprehensively evaluate the subjective, objective, and combined methodologies for assessing AI-generated music. Ultimately, this study provides a valuable reference for unifying generative AI in the field of music evaluation.
  arXiv  Detail & Related papers  (2023-08-26T02:44:33Z)
• MARBLE: Music Audio Representation Benchmark for Universal Evaluation [79.25065218663458]
  We introduce the Music Audio Representation Benchmark for universaL Evaluation, termed MARBLE. It aims to provide a benchmark for various Music Information Retrieval (MIR) tasks by defining a comprehensive taxonomy with four hierarchy levels, including acoustic, performance, score, and high-level description. We then establish a unified protocol based on 14 tasks on 8 public-available datasets, providing a fair and standard assessment of representations of all open-sourced pre-trained models developed on music recordings as baselines.
  arXiv  Detail & Related papers  (2023-06-18T12:56:46Z)
• Simple and Controllable Music Generation [94.61958781346176]
  MusicGen is a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns.
  arXiv  Detail & Related papers  (2023-06-08T15:31:05Z)
• Music Instrument Classification Reprogrammed [79.68916470119743]
  "Reprogramming" is a technique that utilizes pre-trained deep and complex neural networks originally targeting a different task by modifying and mapping both the input and output of the pre-trained model. We demonstrate that reprogramming can effectively leverage the power of the representation learned for a different task and that the resulting reprogrammed system can perform on par or even outperform state-of-the-art systems at a fraction of training parameters.
  arXiv  Detail & Related papers  (2022-11-15T18:26:01Z)
• Sequence Generation using Deep Recurrent Networks and Embeddings: A study case in music [69.2737664640826]
  This paper evaluates different types of memory mechanisms (memory cells) and analyses their performance in the field of music composition. A set of quantitative metrics is presented to evaluate the performance of the proposed architecture automatically.
  arXiv  Detail & Related papers  (2020-12-02T14:19:19Z)
• Research on AI Composition Recognition Based on Music Rules [7.699648754969773]
  Article constructs a music-rule-identifying algorithm through extracting modes. It will identify the stability of the mode of machine-generated music to judge whether it is artificial intelligent.
  arXiv  Detail & Related papers  (2020-10-15T14:51:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.