PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Training

• URL: http://arxiv.org/abs/2407.03361v1
• Date: Wed, 26 Jun 2024 03:35:54 GMT
• Title: PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Training
• Authors: Xiao Liang, Zijian Zhao, Weichao Zeng, Yutong He, Fupeng He, Yiyi Wang, Chengying Gao,
• Abstract summary: PianoBART is a pre-trained model that uses BART for both symbolic piano music generation and understanding. We devise a multi-level object selection strategy for different pre-training tasks of PianoBART, which can prevent information leakage or loss. Experiments demonstrate that PianoBART efficiently learns musical patterns and achieves outstanding performance in generating high-quality coherent pieces.
• Score: 8.484581633133542
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning musical structures and composition patterns is necessary for both music generation and understanding, but current methods do not make uniform use of learned features to generate and comprehend music simultaneously. In this paper, we propose PianoBART, a pre-trained model that uses BART for both symbolic piano music generation and understanding. We devise a multi-level object selection strategy for different pre-training tasks of PianoBART, which can prevent information leakage or loss and enhance learning ability. The musical semantics captured in pre-training are fine-tuned for music generation and understanding tasks. Experiments demonstrate that PianoBART efficiently learns musical patterns and achieves outstanding performance in generating high-quality coherent pieces and comprehending music. Our code and supplementary material are available at https://github.com/RS2002/PianoBart.

Related papers

• Towards Explainable and Interpretable Musical Difficulty Estimation: A Parameter-efficient Approach [49.2787113554916]
  Estimating music piece difficulty is important for organizing educational music collections. Our work employs explainable descriptors for difficulty estimation in symbolic music representations. Our approach, evaluated in piano repertoire categorized in 9 classes, achieved 41.4% accuracy independently, with a mean squared error (MSE) of 1.7.
  arXiv  Detail & Related papers  (2024-08-01T11:23:42Z)
• PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performance [15.21347897534943]
  We construct a piano-hand motion generation benchmark to guide hand movements and fingerings for piano playing. To this end, we collect an annotated dataset, PianoMotion10M, consisting of 116 hours of piano playing videos from a bird's-eye view with 10 million annotated hand poses.
  arXiv  Detail & Related papers  (2024-06-13T17:05:23Z)
• ComposerX: Multi-Agent Symbolic Music Composition with LLMs [51.68908082829048]
  Music composition is a complex task that requires abilities to understand and generate information with long dependency and harmony constraints. Current LLMs easily fail in this task, generating ill-written music even when equipped with modern techniques like In-Context-Learning and Chain-of-Thoughts. We propose ComposerX, an agent-based symbolic music generation framework.
  arXiv  Detail & Related papers  (2024-04-28T06:17:42Z)
• GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework [58.64512825534638]
  Symbolic music generation aims to create musical notes, which can help users compose music. We introduce a framework known as GETMusic, with GET'' standing for GEnerate music Tracks'' GETScore represents musical notes as tokens and organizes tokens in a 2D structure, with tracks stacked vertically and progressing horizontally over time. Our proposed representation, coupled with the non-autoregressive generative model, empowers GETMusic to generate music with any arbitrary source-target track combinations.
  arXiv  Detail & Related papers  (2023-05-18T09:53:23Z)
• Compose & Embellish: Well-Structured Piano Performance Generation via A Two-Stage Approach [36.49582705724548]
  We devise a two-stage Transformer-based framework that Composes a lead sheet first, and then Embellishes it with accompaniment and expressive touches. Our objective and subjective experiments show that Compose & Embellish shrinks the gap in structureness between a current state of the art and real performances by half, and improves other musical aspects such as richness and coherence as well.
  arXiv  Detail & Related papers  (2022-09-17T01:20:59Z)
• Concept-Based Techniques for "Musicologist-friendly" Explanations in a Deep Music Classifier [5.442298461804281]
  We focus on more human-friendly explanations based on high-level musical concepts. Our research targets trained systems (post-hoc explanations) and explores two approaches. We demonstrate both techniques on an existing symbolic composer classification system, showcase their potential, and highlight their intrinsic limitations.
  arXiv  Detail & Related papers  (2022-08-26T07:45:29Z)
• MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training [97.91071692716406]
  Symbolic music understanding refers to the understanding of music from the symbolic data. MusicBERT is a large-scale pre-trained model for music understanding.
  arXiv  Detail & Related papers  (2021-06-10T10:13:05Z)
• Towards Learning to Play Piano with Dexterous Hands and Touch [79.48656721563795]
  We demonstrate how an agent can learn directly from machine-readable music score to play the piano with dexterous hands on a simulated piano. We achieve this by using a touch-augmented reward and a novel curriculum of tasks.
  arXiv  Detail & Related papers  (2021-06-03T17:59:31Z)
• Music Gesture for Visual Sound Separation [121.36275456396075]
  "Music Gesture" is a keypoint-based structured representation to explicitly model the body and finger movements of musicians when they perform music. We first adopt a context-aware graph network to integrate visual semantic context with body dynamics, and then apply an audio-visual fusion model to associate body movements with the corresponding audio signals.
  arXiv  Detail & Related papers  (2020-04-20T17:53:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.