Cluster and Separate: a GNN Approach to Voice and Staff Prediction for Score Engraving

• URL: http://arxiv.org/abs/2407.21030v1
• Date: Mon, 15 Jul 2024 14:36:13 GMT
• Title: Cluster and Separate: a GNN Approach to Voice and Staff Prediction for Score Engraving
• Authors: Francesco Foscarin, Emmanouil Karystinaios, Eita Nakamura, Gerhard Widmer,
• Abstract summary: This paper approaches the problem of separating the notes from a quantized symbolic music piece (e.g., a MIDI file) into multiple voices and staves. We propose an end-to-end system based on graph neural networks that notes that belong to the same chord and connect them with edges if they are part of a voice.
• Score: 5.572472212662453
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper approaches the problem of separating the notes from a quantized symbolic music piece (e.g., a MIDI file) into multiple voices and staves. This is a fundamental part of the larger task of music score engraving (or score typesetting), which aims to produce readable musical scores for human performers. We focus on piano music and support homophonic voices, i.e., voices that can contain chords, and cross-staff voices, which are notably difficult tasks that have often been overlooked in previous research. We propose an end-to-end system based on graph neural networks that clusters notes that belong to the same chord and connects them with edges if they are part of a voice. Our results show clear and consistent improvements over a previous approach on two datasets of different styles. To aid the qualitative analysis of our results, we support the export in symbolic music formats and provide a direct visualization of our outputs graph over the musical score. All code and pre-trained models are available at https://github.com/CPJKU/piano_svsep

Related papers

• Simple and Controllable Music Generation [94.61958781346176]
  MusicGen is a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns.
  arXiv  Detail & Related papers  (2023-06-08T15:31:05Z)
• GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework [58.64512825534638]
  Symbolic music generation aims to create musical notes, which can help users compose music. We introduce a framework known as GETMusic, with GET'' standing for GEnerate music Tracks'' GETScore represents musical notes as tokens and organizes tokens in a 2D structure, with tracks stacked vertically and progressing horizontally over time. Our proposed representation, coupled with the non-autoregressive generative model, empowers GETMusic to generate music with any arbitrary source-target track combinations.
  arXiv  Detail & Related papers  (2023-05-18T09:53:23Z)
• RMSSinger: Realistic-Music-Score based Singing Voice Synthesis [56.51475521778443]
  RMS-SVS aims to generate high-quality singing voices given realistic music scores with different note types. We propose RMSSinger, the first RMS-SVS method, which takes realistic music scores as input. In RMSSinger, we introduce word-level modeling to avoid the time-consuming phoneme duration annotation and the complicated phoneme-level mel-note alignment.
  arXiv  Detail & Related papers  (2023-05-18T03:57:51Z)
• Musical Voice Separation as Link Prediction: Modeling a Musical Perception Task as a Multi-Trajectory Tracking Problem [6.617487928813374]
  This paper targets the perceptual task of separating the different interacting voices, i.e., monophonic melodic streams, in a polyphonic musical piece. We model this task as a Multi-Trajectory Tracking (MTT) problem from discrete observations, i.e. notes in a pitch-time space. Our approach builds a graph from a musical piece, by creating one node for every note, and separates the melodic trajectories by predicting a link between two notes if they are consecutive in the same voice/stream.
  arXiv  Detail & Related papers  (2023-04-28T13:48:00Z)
• A Phoneme-Informed Neural Network Model for Note-Level Singing Transcription [11.951441023641975]
  We propose a method of finding note onsets of singing voice more accurately by leveraging the linguistic characteristics of singing. Our approach substantially improves the performance of singing transcription and emphasizes the importance of linguistic features in singing analysis.
  arXiv  Detail & Related papers  (2023-04-12T15:36:01Z)
• Melody transcription via generative pre-training [86.08508957229348]
  Key challenge in melody transcription is building methods which can handle broad audio containing any number of instrument ensembles and musical styles. To confront this challenge, we leverage representations from Jukebox (Dhariwal et al. 2020), a generative model of broad music audio. We derive a new dataset containing $50$ hours of melody transcriptions from crowdsourced annotations of broad music.
  arXiv  Detail & Related papers  (2022-12-04T18:09:23Z)
• Cadence Detection in Symbolic Classical Music using Graph Neural Networks [7.817685358710508]
  We present a graph representation of symbolic scores as an intermediate means to solve the cadence detection task. We approach cadence detection as an imbalanced node classification problem using a Graph Convolutional Network. Our experiments suggest that graph convolution can learn non-local features that assist in cadence detection, freeing us from the need of having to devise specialized features that encode non-local context.
  arXiv  Detail & Related papers  (2022-08-31T12:39:57Z)
• Score Transformer: Generating Musical Score from Note-level Representation [2.3554584457413483]
  We train the Transformer model to transcribe note-level representation into appropriate music notation. We also explore an effective notation-level token representation to work with the model.
  arXiv  Detail & Related papers  (2021-12-01T09:08:01Z)
• From Note-Level to Chord-Level Neural Network Models for Voice Separation in Symbolic Music [0.0]
  We train neural networks that assign notes to voices either separately for each note in a chord (note-level), or jointly to all notes in a chord (chord-level) Both models surpass a strong baseline based on an iterative application of an envelope extraction function. The two models are also shown to outperform previous approaches on separating the voices in Bach music.
  arXiv  Detail & Related papers  (2020-11-05T18:39:42Z)
• Melody-Conditioned Lyrics Generation with SeqGANs [81.2302502902865]
  We propose an end-to-end melody-conditioned lyrics generation system based on Sequence Generative Adversarial Networks (SeqGAN) We show that the input conditions have no negative impact on the evaluation metrics while enabling the network to produce more meaningful results.
  arXiv  Detail & Related papers  (2020-10-28T02:35:40Z)
• Music Gesture for Visual Sound Separation [121.36275456396075]
  "Music Gesture" is a keypoint-based structured representation to explicitly model the body and finger movements of musicians when they perform music. We first adopt a context-aware graph network to integrate visual semantic context with body dynamics, and then apply an audio-visual fusion model to associate body movements with the corresponding audio signals.
  arXiv  Detail & Related papers  (2020-04-20T17:53:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.