Semantic-Aware Interpretable Multimodal Music Auto-Tagging

• URL: http://arxiv.org/abs/2505.17233v2
• Date: Mon, 26 May 2025 09:40:25 GMT
• Title: Semantic-Aware Interpretable Multimodal Music Auto-Tagging
• Authors: Andreas Patakis, Vassilis Lyberatos, Spyridon Kantarelis, Edmund Dervakos, Giorgos Stamou,
• Abstract summary: We present an interpretable framework for music auto-tagging that leverages groups of musically meaningful multimodal features.<n>Our method achieves competitive tagging performance while offering a deeper understanding of the decision-making process.
• Score: 1.8541450825478398
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Music auto-tagging is essential for organizing and discovering music in extensive digital libraries. While foundation models achieve exceptional performance in this domain, their outputs often lack interpretability, limiting trust and usability for researchers and end-users alike. In this work, we present an interpretable framework for music auto-tagging that leverages groups of musically meaningful multimodal features, derived from signal processing, deep learning, ontology engineering, and natural language processing. To enhance interpretability, we cluster features semantically and employ an expectation maximization algorithm, assigning distinct weights to each group based on its contribution to the tagging process. Our method achieves competitive tagging performance while offering a deeper understanding of the decision-making process, paving the way for more transparent and user-centric music tagging systems.

Related papers

• Bridging the Gap Between Semantic and User Preference Spaces for Multi-modal Music Representation Learning [10.558648773612191]
  We propose a novel Hierarchical Two-stage Contrastive Learning (HTCL) method that models similarity from the semantic perspective to the user perspective hierarchically.<n>We devise a scalable audio encoder and leverage a pre-trained BERT model as the text encoder to learn audio-text semantics via large-scale contrastive pre-training.
  arXiv  Detail & Related papers  (2025-05-29T09:50:07Z)
• Unifying Multitrack Music Arrangement via Reconstruction Fine-Tuning and Efficient Tokenization [10.714947060480426]
  This paper introduces an efficient multitrack music tokenizer for unconditional and conditional symbolic music generation.<n>A unified sequence-to-sequence reconstruction fine-tuning objective for pre-trained music LMs balances task-specific needs with coherence constraints.<n>Our approach achieves state-of-the-art results on band arrangement, piano reduction, and drum arrangement, surpassing task-specific models in both objective metrics and perceptual quality.
  arXiv  Detail & Related papers  (2024-08-27T16:18:51Z)
• Towards Explainable and Interpretable Musical Difficulty Estimation: A Parameter-efficient Approach [49.2787113554916]
  Estimating music piece difficulty is important for organizing educational music collections. Our work employs explainable descriptors for difficulty estimation in symbolic music representations. Our approach, evaluated in piano repertoire categorized in 9 classes, achieved 41.4% accuracy independently, with a mean squared error (MSE) of 1.7.
  arXiv  Detail & Related papers  (2024-08-01T11:23:42Z)
• ComposerX: Multi-Agent Symbolic Music Composition with LLMs [51.68908082829048]
  Music composition is a complex task that requires abilities to understand and generate information with long dependency and harmony constraints. Current LLMs easily fail in this task, generating ill-written music even when equipped with modern techniques like In-Context-Learning and Chain-of-Thoughts. We propose ComposerX, an agent-based symbolic music generation framework.
  arXiv  Detail & Related papers  (2024-04-28T06:17:42Z)
• Perceptual Musical Features for Interpretable Audio Tagging [2.1730712607705485]
  This study explores the relevance of interpretability in the context of automatic music tagging. We constructed a workflow that incorporates three different information extraction techniques. We conducted experiments on two datasets, namely the MTG-Jamendo dataset and the GTZAN dataset.
  arXiv  Detail & Related papers  (2023-12-18T14:31:58Z)
• MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models [54.55063772090821]
  MusicAgent integrates numerous music-related tools and an autonomous workflow to address user requirements. The primary goal of this system is to free users from the intricacies of AI-music tools, enabling them to concentrate on the creative aspect.
  arXiv  Detail & Related papers  (2023-10-18T13:31:10Z)
• Exploring the Efficacy of Pre-trained Checkpoints in Text-to-Music Generation Task [86.72661027591394]
  We generate complete and semantically consistent symbolic music scores from text descriptions. We explore the efficacy of using publicly available checkpoints for natural language processing in the task of text-to-music generation. Our experimental results show that the improvement from using pre-trained checkpoints is statistically significant in terms of BLEU score and edit distance similarity.
  arXiv  Detail & Related papers  (2022-11-21T07:19:17Z)
• Towards Context-Aware Neural Performance-Score Synchronisation [2.0305676256390934]
  Music synchronisation provides a way to navigate among multiple representations of music in a unified manner. Traditional synchronisation methods compute alignment using knowledge-driven and performance analysis approaches. This PhD furthers the development of performance-score synchronisation research by proposing data-driven, context-aware alignment approaches.
  arXiv  Detail & Related papers  (2022-05-31T16:45:25Z)
• Learning music audio representations via weak language supervision [14.335950077921435]
  We design a multimodal architecture for music and language pre-training (MuLaP) optimised via a set of proxy tasks. weak supervision is provided in the form of noisy natural language descriptions conveying the overall musical content of the track. We demonstrate the usefulness of our approach by comparing the performance of audio representations produced by the same audio backbone with different training strategies.
  arXiv  Detail & Related papers  (2021-12-08T10:30:52Z)
• Sequence Generation using Deep Recurrent Networks and Embeddings: A study case in music [69.2737664640826]
  This paper evaluates different types of memory mechanisms (memory cells) and analyses their performance in the field of music composition. A set of quantitative metrics is presented to evaluate the performance of the proposed architecture automatically.
  arXiv  Detail & Related papers  (2020-12-02T14:19:19Z)
• Music Gesture for Visual Sound Separation [121.36275456396075]
  "Music Gesture" is a keypoint-based structured representation to explicitly model the body and finger movements of musicians when they perform music. We first adopt a context-aware graph network to integrate visual semantic context with body dynamics, and then apply an audio-visual fusion model to associate body movements with the corresponding audio signals.
  arXiv  Detail & Related papers  (2020-04-20T17:53:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.