Adopting State-of-the-Art Pretrained Audio Representations for Music Recommender Systems

• URL: http://arxiv.org/abs/2604.23077v1
• Date: Sat, 25 Apr 2026 00:09:58 GMT
• Title: Adopting State-of-the-Art Pretrained Audio Representations for Music Recommender Systems
• Abstract summary: Music Information Retrieval (MIR) research community has released various models pretrained on large amounts of music data.<n>Music FM, Music2Vec, MERT, EncodecMAE, Jukebox, MusiCNN, MULE, MuQ and MuQ-MuLan are investigated.<n>We find that pretrained audio representations exhibit significant performance disparity between traditional MIR tasks and both hot and cold music recommendations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Over the years, Music Information Retrieval (MIR) research community has released various models pretrained on large amounts of music data. Transfer learning showcases the proven effectiveness of pretrained backend models for a broad spectrum of downstream tasks, including auto-tagging and genre classification. However, MIR papers generally do not explore the efficiency of pretrained models for Music Recommender Systems (MRS). In addition, the Recommender Systems community tends to favour traditional end-to-end neural network training. Our research addresses this gap and evaluates the performance of nine pretrained backend models (MusicFM, Music2Vec, MERT, EncodecMAE, Jukebox, MusiCNN, MULE, MuQ and MuQ-MuLan) in the context of MRS. We assess them using five recommendation approaches: K-Nearest Neighbours (KNN), Shallow Neural Network, Contrastive Multi-Modal projection, a Hybrid model, and BERT4Rec both for the hot and cold-start scenarios. Our findings suggest that pretrained audio representations exhibit significant performance disparity between traditional MIR tasks and both hot and cold music recommendations, indicating that valuable aspects of musical information captured by backend models may differ depending on the task. This study establishes a foundation for further exploration of pretrained audio representations to enhance music recommendation systems.

Related papers

• From Classification to Recommendation: Empirical Analysis of Audio Embedding Models Application for Content-Based Music Recommendation [21.60252248417606]
  We evaluate six representative audio encoders across three types of music recommender systems.<n>We find that interaction-based sequential training substantially reduces performance differences among encoders.<n>We also find that increasing Semantic-ID capacity does not consistently improve generative recommender systems.
  arXiv  Detail & Related papers  (2026-08-07T08:03:29Z)
• Revisiting Content-Based Music Recommendation: Efficient Feature Aggregation from Large-Scale Music Models [54.4270504928356]
  Music Recommendation Systems (MRSs) are a cornerstone of modern streaming platforms.<n>We propose TASTE, a comprehensive dataset and benchmarking framework designed to highlight the role of multimodal information in music recommendation.<n>By leveraging recent large-scale self-supervised music encoders, we demonstrate the substantial value of the extracted audio representations across recommendation tasks.
  arXiv  Detail & Related papers  (2026-02-10T15:24:41Z)
• "Beyond the past": Leveraging Audio and Human Memory for Sequential Music Recommendation [6.875744149600454]
  On music streaming services, listening sessions are often composed of a balance of familiar and new tracks.<n>We propose a model that leverages audio information to predict in advance the activation of new tracks.
  arXiv  Detail & Related papers  (2025-07-23T09:37:23Z)
• Towards Leveraging Contrastively Pretrained Neural Audio Embeddings for Recommender Tasks [18.95453617434051]
  Music recommender systems frequently utilize network-based models to capture relationships between music pieces, artists, and users. New music pieces or artists often face the cold-start problem due to insufficient initial information. To address this, one can extract content-based information directly from the music to enhance collaborative-filtering-based methods.
  arXiv  Detail & Related papers  (2024-09-13T17:53:06Z)
• Comparative Analysis of Pretrained Audio Representations in Music Recommender Systems [0.0]
  Music Information Retrieval (MIR) has proposed various models pretrained on large amounts of music data. transfer learning showcases the proven effectiveness of pretrained backend models with a broad spectrum of downstream tasks. Music Recommender Systems tend to favour traditional end-to-end neural network learning over pretrained models.
  arXiv  Detail & Related papers  (2024-09-13T17:03:56Z)
• Foundation Models for Music: A Survey [77.77088584651268]
  Foundations models (FMs) have profoundly impacted diverse sectors, including music. This comprehensive review examines state-of-the-art (SOTA) pre-trained models and foundation models in music.
  arXiv  Detail & Related papers  (2024-08-26T15:13:14Z)
• MuPT: A Generative Symbolic Music Pretrained Transformer [56.09299510129221]
  We explore the application of Large Language Models (LLMs) to the pre-training of music. To address the challenges associated with misaligned measures from different tracks during generation, we propose a Synchronized Multi-Track ABC Notation (SMT-ABC Notation) Our contributions include a series of models capable of handling up to 8192 tokens, covering 90% of the symbolic music data in our training set.
  arXiv  Detail & Related papers  (2024-04-09T15:35:52Z)
• MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training [74.32603591331718]
  We propose an acoustic Music undERstanding model with large-scale self-supervised Training (MERT), which incorporates teacher models to provide pseudo labels in the masked language modelling (MLM) style acoustic pre-training.<n> Experimental results indicate that our model can generalise and perform well on 14 music understanding tasks and attain state-of-the-art (SOTA) overall scores.
  arXiv  Detail & Related papers  (2023-05-31T18:27:43Z)
• Contrastive Audio-Visual Masked Autoencoder [85.53776628515561]
  Contrastive Audio-Visual Masked Auto-Encoder (CAV-MAE) Our fully self-supervised pretrained CAV-MAE achieves a new SOTA accuracy of 65.9% on VGGSound.
  arXiv  Detail & Related papers  (2022-10-02T07:29:57Z)
• Codified audio language modeling learns useful representations for music information retrieval [77.63657430536593]
  We show that language models pre-trained on codified (discretely-encoded) music audio learn representations that are useful for downstream MIR tasks. To determine if Jukebox's representations contain useful information for MIR, we use them as input features to train shallow models on several MIR tasks. We observe that representations from Jukebox are considerably stronger than those from models pre-trained on tagging, suggesting that pre-training via codified audio language modeling may address blind spots in conventional approaches.
  arXiv  Detail & Related papers  (2021-07-12T18:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.