Scaling Self-Supervised Representation Learning for Symbolic Piano Performance

• URL: http://arxiv.org/abs/2506.23869v1
• Date: Mon, 30 Jun 2025 14:00:14 GMT
• Title: Scaling Self-Supervised Representation Learning for Symbolic Piano Performance
• Authors: Louis Bradshaw, Honglu Fan, Alexander Spangher, Stella Biderman, Simon Colton,
• Abstract summary: We study the capabilities of generative autoregressive transformer models trained on large amounts of symbolic solo-piano transcriptions.<n>We use a comparatively smaller, high-quality subset to finetune models to produce musical continuations, perform symbolic classification tasks, and produce general-purpose contrastive MIDI embeddings.
• Score: 52.661197827466886
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the capabilities of generative autoregressive transformer models trained on large amounts of symbolic solo-piano transcriptions. After first pretraining on approximately 60,000 hours of music, we use a comparatively smaller, high-quality subset, to finetune models to produce musical continuations, perform symbolic classification tasks, and produce general-purpose contrastive MIDI embeddings by adapting the SimCLR framework to symbolic music. When evaluating piano continuation coherence, our generative model outperforms leading symbolic generation techniques and remains competitive with proprietary audio generation models. On MIR classification benchmarks, frozen representations from our contrastive model achieve state-of-the-art results in linear probe experiments, while direct finetuning demonstrates the generalizability of pretrained representations, often requiring only a few hundred labeled examples to specialize to downstream tasks.

Related papers

• MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization [24.991558192161]
  We propose a self-supervised music representation learning model for music understanding.<n>MuQ is trained to predict tokens generated by Mel Residual Vector Quantization (Mel-RVQ)<n>Experiments in a large variety of downstream tasks demonstrate that MuQ outperforms previous self-supervised music representation models.
  arXiv  Detail & Related papers  (2025-01-02T07:08:29Z)
• Generative Pre-training for Speech with Flow Matching [81.59952572752248]
  We pre-trained a generative model, named SpeechFlow, on 60k hours of untranscribed speech with Flow Matching and masked conditions. Experiment results show the pre-trained generative model can be fine-tuned with task-specific data to match or surpass existing expert models on speech enhancement, separation, and synthesis.
  arXiv  Detail & Related papers  (2023-10-25T03:40:50Z)
• Performance Conditioning for Diffusion-Based Multi-Instrument Music Synthesis [15.670399197114012]
  We propose enhancing control of multi-instrument synthesis by conditioning a generative model on a specific performance and recording environment. Performance conditioning is a tool indicating the generative model to synthesize music with style and timbre of specific instruments taken from specific performances. Our prototype is evaluated using uncurated performances with diverse instrumentation and state-of-the-art FAD realism scores.
  arXiv  Detail & Related papers  (2023-09-21T17:44:57Z)
• Simple and Controllable Music Generation [94.61958781346176]
  MusicGen is a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns.
  arXiv  Detail & Related papers  (2023-06-08T15:31:05Z)
• MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training [74.32603591331718]
  We propose an acoustic Music undERstanding model with large-scale self-supervised Training (MERT), which incorporates teacher models to provide pseudo labels in the masked language modelling (MLM) style acoustic pre-training.<n> Experimental results indicate that our model can generalise and perform well on 14 music understanding tasks and attain state-of-the-art (SOTA) overall scores.
  arXiv  Detail & Related papers  (2023-05-31T18:27:43Z)
• SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data [78.21197488065177]
  Recent success in fine-tuning large models, that are pretrained on broad data at scale, on downstream tasks has led to a significant paradigm shift in deep learning. This paper proposes a new task-agnostic framework, textitSynBench, to measure the quality of pretrained representations using synthetic data.
  arXiv  Detail & Related papers  (2022-10-06T15:25:00Z)
• BERT-like Pre-training for Symbolic Piano Music Classification Tasks [15.02723006489356]
  This article presents a benchmark study of symbolic piano music classification using the Bidirectional Representations from Transformers (BERT) approach. We pre-train two 12-layer Transformer models using the BERT approach and fine-tune them for four downstream classification tasks. Our evaluation shows that the BERT approach leads to higher classification accuracy than recurrent neural network (RNN)-based baselines.
  arXiv  Detail & Related papers  (2021-07-12T07:03:57Z)
• DiffSinger: Diffusion Acoustic Model for Singing Voice Synthesis [53.19363127760314]
  DiffSinger is a parameterized Markov chain which iteratively converts the noise into mel-spectrogram conditioned on the music score. The evaluations conducted on the Chinese singing dataset demonstrate that DiffSinger outperforms state-of-the-art SVS work with a notable margin.
  arXiv  Detail & Related papers  (2021-05-06T05:21:42Z)
• Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling [5.88864611435337]
  We present a framework that can learn high-level feature representations with a limited amount of data. We refer to our proposed framework as Music FaderNets, which is inspired by the fact that low-level attributes can be continuously manipulated. We demonstrate that the model successfully learns the intrinsic relationship between arousal and its corresponding low-level attributes.
  arXiv  Detail & Related papers  (2020-07-29T16:01:45Z)
• Generative Modelling for Controllable Audio Synthesis of Expressive Piano Performance [6.531546527140474]
  controllable neural audio synthesizer based on Gaussian Mixture Variational Autoencoders (GM-VAE) We demonstrate how the model is able to apply fine-grained style morphing over the course of the audio.
  arXiv  Detail & Related papers  (2020-06-16T12:54:41Z)
• Continuous Melody Generation via Disentangled Short-Term Representations and Structural Conditions [14.786601824794369]
  We present a model for composing melodies given a user specified symbolic scenario combined with a previous music context. Our model is capable of generating long melodies by regarding 8-beat note sequences as basic units, and shares consistent rhythm pattern structure with another specific song. Results show that the music generated by our model tends to have salient repetition structures, rich motives, and stable rhythm patterns.
  arXiv  Detail & Related papers  (2020-02-05T06:23:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.