PianoKontext: Expressive Performance Rendering from Deadpan Context

• URL: http://arxiv.org/abs/2606.12282v1
• Date: Wed, 10 Jun 2026 16:20:24 GMT
• Title: PianoKontext: Expressive Performance Rendering from Deadpan Context
• Abstract summary: PianoKontext is a flow matching rendering model for classical piano music.<n>It generates variable-length performances in the latent space of a pretrained Music2Latent model.
• Score: 0.009905207929946798
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Expressive performance rendering (EPR) aims to generate realistic performances constrained on sequences of notes. However, flow matching audio editing models manipulate only synchronized music samples of the same duration, limiting their understanding of expressive timing. We introduce PianoKontext, a flow matching rendering model for classical piano music that generates variable-length performances in the latent space of a pretrained Music2Latent model. We synthesize MIDI scores into deadpan audio and employ Dynamic Time Warping (DTW) in the latent space to construct paired data for training. The aligned embeddings are concatenated in DiT blocks, allowing for a simple and effective learning of the dependencies between the score and performances. Audio samples are available at our demo page: https://realfolkcode.github.io/pianokontext_demo/.

Related papers

• FIGMA: Towards FIne-Grained Music retrievAl [65.98380295254817]
  Retrieving music using natural language descriptions has improved with contrastive audio-text models such as CLAP, but current systems remain limited to coarse semantic queries.<n>We show that this limitation stems from the contrastive learning objective itself: despite being trained on long captions, CLAP-based models effectively utilize only the first few tokens.<n>We propose FIGMA, a multi-view contrastive architecture that addresses this limitation by jointly optimizing global audio-text alignment and frame-level, token-wise alignment.
  arXiv  Detail & Related papers  (2026-06-04T18:05:39Z)
• Rubato: Transcribing Piano Music with Timestamps [47.76733897717644]
  We consider the conversion of musical recordings into human-readable sheet music annotated with timestamps.<n>We introduce (1) a prompt-conditioned encoder-decoder model, named Rubato, trained to output (2) a new textual representation for polyphonic music, named InterMo.<n>Our experiments demonstrate that Rubato produces timestamped piano sheet music from audio with higher notational accuracy than the best existing approaches.
  arXiv  Detail & Related papers  (2026-05-22T23:47:21Z)
• Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training [26.885642751756695]
  Pianist Transformer is a unified Musical Instrument Digital Interface (MIDI) data representation for learning the shared principles of musical structure and expression without explicit annotation.<n>It achieves strong objective metrics and human-level subjective ratings.<n>Overall, Pianist Transformer establishes a scalable path toward human-like performance synthesis in the music domain.
  arXiv  Detail & Related papers  (2025-12-02T11:13:29Z)
• InstructAudio: Unified speech and music generation with natural language instruction [52.76518112649456]
  InstructAudio is a unified framework that enables instruction-based control of acoustic attributes.<n>It supports expressive speech, music, and dialogue generation in English and Chinese.
  arXiv  Detail & Related papers  (2025-11-23T15:15:21Z)
• SyMuPe: Affective and Controllable Symbolic Music Performance [0.00746020873338928]
  We present SyMuPe, a novel framework for developing and training affective and controllable piano performance models.<n>Our flagship model, PianoFlow, uses conditional flow matching trained to solve diverse multi-mask performance inpainting tasks.<n>For emotion control, we present and analyze samples generated under different text conditioning scenarios.
  arXiv  Detail & Related papers  (2025-11-05T12:42:08Z)
• PianoVAM: A Multimodal Piano Performance Dataset [56.318475235705954]
  PianoVAM is a comprehensive piano performance dataset that includes videos, audio, MIDI, hand landmarks, fingering labels, and rich metadata.<n>The dataset was recorded using a Disklavier piano, capturing audio and MIDI from amateur pianists during their daily practice sessions.<n>Hand landmarks and fingering labels were extracted using a pretrained hand pose estimation model and a semi-automated fingering annotation algorithm.
  arXiv  Detail & Related papers  (2025-09-10T17:35:58Z)
• MIDI-VALLE: Improving Expressive Piano Performance Synthesis Through Neural Codec Language Modelling [32.78044321881271]
  We propose MIDI-VALLE, a neural language model adapted from the VALLE framework for personalised text-to-speech synthesis.<n>VALLE encodes both MIDI and audio as discrete tokens, facilitating a more consistent and robust modelling of piano performances.<n> Evaluation results show that MIDI-VALLE significantly outperforms a state-of-the-art baseline.
  arXiv  Detail & Related papers  (2025-07-11T12:28:20Z)
• Accompanied Singing Voice Synthesis with Fully Text-controlled Melody [61.147446955297625]
  Text-to-song (TTSong) is a music generation task that synthesizes accompanied singing voices. We present MelodyLM, the first TTSong model that generates high-quality song pieces with fully text-controlled melodies.
  arXiv  Detail & Related papers  (2024-07-02T08:23:38Z)
• RMSSinger: Realistic-Music-Score based Singing Voice Synthesis [56.51475521778443]
  RMS-SVS aims to generate high-quality singing voices given realistic music scores with different note types. We propose RMSSinger, the first RMS-SVS method, which takes realistic music scores as input. In RMSSinger, we introduce word-level modeling to avoid the time-consuming phoneme duration annotation and the complicated phoneme-level mel-note alignment.
  arXiv  Detail & Related papers  (2023-05-18T03:57:51Z)
• Generative Modelling for Controllable Audio Synthesis of Expressive Piano Performance [6.531546527140474]
  controllable neural audio synthesizer based on Gaussian Mixture Variational Autoencoders (GM-VAE) We demonstrate how the model is able to apply fine-grained style morphing over the course of the audio.
  arXiv  Detail & Related papers  (2020-06-16T12:54:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.