A Comprehensive Survey for Evaluation Methodologies of AI-Generated
Music
- URL: http://arxiv.org/abs/2308.13736v1
- Date: Sat, 26 Aug 2023 02:44:33 GMT
- Title: A Comprehensive Survey for Evaluation Methodologies of AI-Generated
Music
- Authors: Zeyu Xiong, Weitao Wang, Jing Yu, Yue Lin, Ziyan Wang
- Abstract summary: This study aims to comprehensively evaluate the subjective, objective, and combined methodologies for assessing AI-generated music.
Ultimately, this study provides a valuable reference for unifying generative AI in the field of music evaluation.
- Score: 14.453416870193072
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In recent years, AI-generated music has made significant progress, with
several models performing well in multimodal and complex musical genres and
scenes. While objective metrics can be used to evaluate generative music, they
often lack interpretability for musical evaluation. Therefore, researchers
often resort to subjective user studies to assess the quality of the generated
works, which can be resource-intensive and less reproducible than objective
metrics. This study aims to comprehensively evaluate the subjective, objective,
and combined methodologies for assessing AI-generated music, highlighting the
advantages and disadvantages of each approach. Ultimately, this study provides
a valuable reference for unifying generative AI in the field of music
evaluation.
Related papers
- Towards Assessing Data Replication in Music Generation with Music Similarity Metrics on Raw Audio [25.254669525489923]
We present the Music Replication Assessment (MiRA) tool: a model-independent open evaluation method based on diverse audio music similarity metrics.
We evaluate the ability of five metrics to identify exact replication, by conducting a controlled replication experiment in different music genres based on synthetic samples.
Our results show that the proposed methodology can estimate exact data replication with a proportion higher than 10%.
arXiv Detail & Related papers (2024-07-19T14:52:11Z) - Deep Learning-Based Object Pose Estimation: A Comprehensive Survey [73.74933379151419]
We discuss the recent advances in deep learning-based object pose estimation.
Our survey also covers multiple input data modalities, degrees-of-freedom of output poses, object properties, and downstream tasks.
arXiv Detail & Related papers (2024-05-13T14:44:22Z) - A Survey of Music Generation in the Context of Interaction [3.6522809408725223]
Machine learning has been successfully used to compose and generate music, both melodies and polyphonic pieces.
Most of these models are not suitable for human-machine co-creation through live interaction.
arXiv Detail & Related papers (2024-02-23T12:41:44Z) - MARBLE: Music Audio Representation Benchmark for Universal Evaluation [79.25065218663458]
We introduce the Music Audio Representation Benchmark for universaL Evaluation, termed MARBLE.
It aims to provide a benchmark for various Music Information Retrieval (MIR) tasks by defining a comprehensive taxonomy with four hierarchy levels, including acoustic, performance, score, and high-level description.
We then establish a unified protocol based on 14 tasks on 8 public-available datasets, providing a fair and standard assessment of representations of all open-sourced pre-trained models developed on music recordings as baselines.
arXiv Detail & Related papers (2023-06-18T12:56:46Z) - An Order-Complexity Model for Aesthetic Quality Assessment of Symbolic
Homophony Music Scores [8.751312368054016]
The quality of music score generated by AI is relatively poor compared with that created by human composers.
This paper proposes an objective quantitative evaluation method for homophony music score aesthetic quality assessment.
arXiv Detail & Related papers (2023-01-14T12:30:16Z) - Exploring the Efficacy of Pre-trained Checkpoints in Text-to-Music
Generation Task [86.72661027591394]
We generate complete and semantically consistent symbolic music scores from text descriptions.
We explore the efficacy of using publicly available checkpoints for natural language processing in the task of text-to-music generation.
Our experimental results show that the improvement from using pre-trained checkpoints is statistically significant in terms of BLEU score and edit distance similarity.
arXiv Detail & Related papers (2022-11-21T07:19:17Z) - Evaluating generative audio systems and their metrics [80.97828572629093]
This paper investigates state-of-the-art approaches side-by-side with (i) a set of previously proposed objective metrics for audio reconstruction, and (ii) a listening study.
Results indicate that currently used objective metrics are insufficient to describe the perceptual quality of current systems.
arXiv Detail & Related papers (2022-08-31T21:48:34Z) - Subjective Evaluation of Deep Learning Models for Symbolic Music
Composition [1.1677169430445211]
We propose a subjective method to evaluate AI-based music composition systems.
We ask questions related to basic music principles to different levels of users based on their musical experience and knowledge.
arXiv Detail & Related papers (2022-03-28T10:56:55Z) - Evaluating Deep Music Generation Methods Using Data Augmentation [13.72212417973239]
We focus on a homogeneous, objective framework for evaluating samples of algorithmically generated music.
We do not seek to assess the musical merit of generated music, but instead explore whether generated samples contain meaningful information pertaining to emotion or mood/theme.
arXiv Detail & Related papers (2021-12-31T20:35:46Z) - Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy
Evaluation Approach [84.02388020258141]
We propose a new framework named ENIGMA for estimating human evaluation scores based on off-policy evaluation in reinforcement learning.
ENIGMA only requires a handful of pre-collected experience data, and therefore does not involve human interaction with the target policy during the evaluation.
Our experiments show that ENIGMA significantly outperforms existing methods in terms of correlation with human evaluation scores.
arXiv Detail & Related papers (2021-02-20T03:29:20Z) - Multi-Modal Music Information Retrieval: Augmenting Audio-Analysis with
Visual Computing for Improved Music Video Analysis [91.3755431537592]
This thesis combines audio-analysis with computer vision to approach Music Information Retrieval (MIR) tasks from a multi-modal perspective.
The main hypothesis of this work is based on the observation that certain expressive categories such as genre or theme can be recognized on the basis of the visual content alone.
The experiments are conducted for three MIR tasks Artist Identification, Music Genre Classification and Cross-Genre Classification.
arXiv Detail & Related papers (2020-02-01T17:57:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.