BEAT: Tokenizing and Generating Symbolic Music by Uniform Temporal Steps

• URL: http://arxiv.org/abs/2604.19532v1
• Date: Tue, 21 Apr 2026 14:53:10 GMT
• Title: BEAT: Tokenizing and Generating Symbolic Music by Uniform Temporal Steps
• Abstract summary: We consider whether an alternative tokenization is possible, where a uniform-length musical step serves as the basic unit.<n>We encode all events within a single time step at the same pitch as one token, and group tokens explicitly by time step.<n>Results show improved musical quality and structural coherence, while additional analyses confirm higher efficiency and more effective capture of long-range patterns.
• Score: 6.961101489217409
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tokenizing music to fit the general framework of language models is a compelling challenge, especially considering the diverse symbolic structures in which music can be represented (e.g., sequences, grids, and graphs). To date, most approaches tokenize symbolic music as sequences of musical events, such as onsets, pitches, time shifts, or compound note events. This strategy is intuitive and has proven effective in Transformer-based models, but it treats the regularity of musical time implicitly: individual tokens may span different durations, resulting in non-uniform time progression. In this paper, we instead consider whether an alternative tokenization is possible, where a uniform-length musical step (e.g., a beat) serves as the basic unit. Specifically, we encode all events within a single time step at the same pitch as one token, and group tokens explicitly by time step, which resembles a sparse encoding of a piano-roll representation. We evaluate the proposed tokenization on music continuation and accompaniment generation tasks, comparing it with mainstream event-based methods. Results show improved musical quality and structural coherence, while additional analyses confirm higher efficiency and more effective capture of long-range patterns with the proposed tokenization.

Related papers

• FIGMA: Towards FIne-Grained Music retrievAl [65.98380295254817]
  Retrieving music using natural language descriptions has improved with contrastive audio-text models such as CLAP, but current systems remain limited to coarse semantic queries.<n>We show that this limitation stems from the contrastive learning objective itself: despite being trained on long captions, CLAP-based models effectively utilize only the first few tokens.<n>We propose FIGMA, a multi-view contrastive architecture that addresses this limitation by jointly optimizing global audio-text alignment and frame-level, token-wise alignment.
  arXiv  Detail & Related papers  (2026-06-04T18:05:39Z)
• Rubato: Transcribing Piano Music with Timestamps [47.76733897717644]
  We consider the conversion of musical recordings into human-readable sheet music annotated with timestamps.<n>We introduce (1) a prompt-conditioned encoder-decoder model, named Rubato, trained to output (2) a new textual representation for polyphonic music, named InterMo.<n>Our experiments demonstrate that Rubato produces timestamped piano sheet music from audio with higher notational accuracy than the best existing approaches.
  arXiv  Detail & Related papers  (2026-05-22T23:47:21Z)
• Evaluating Interval-based Tokenization for Pitch Representation in Symbolic Music Analysis [0.10241134756773229]
  We introduce a general framework for building interval-based tokenizations.<n>We show that interval-based tokenizations improve model performances and facilitate their explainability.
  arXiv  Detail & Related papers  (2025-01-08T17:22:03Z)
• PerceiverS: A Multi-Scale Perceiver with Effective Segmentation for Long-Term Expressive Symbolic Music Generation [4.101665207455494]
  PerceiverS (Segmentation and Scale) is a novel architecture designed to generate long-structured and expressive music.<n>Our approach enhances symbolic music generation by simultaneously learning long-term structural dependencies and short-term expressive details.<n>The proposed model has been evaluated using the Maestro dataset and has demonstrated improvements in generating coherent and diverse music.
  arXiv  Detail & Related papers  (2024-11-13T03:14:10Z)
• Impact of time and note duration tokenizations on deep learning symbolic music modeling [0.0]
  We analyze the common tokenization methods and experiment with time and note duration representations. We demonstrate that explicit information leads to better results depending on the task.
  arXiv  Detail & Related papers  (2023-10-12T16:56:37Z)
• Simple and Controllable Music Generation [94.61958781346176]
  MusicGen is a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns.
  arXiv  Detail & Related papers  (2023-06-08T15:31:05Z)
• From Words to Music: A Study of Subword Tokenization Techniques in Symbolic Music Generation [1.9188864062289432]
  Subword tokenization has been widely successful in text-based natural language processing tasks with Transformer-based models. We apply subword tokenization on post-musical tokenization schemes and find that it enables the generation of longer songs at the same time. Our study suggests that subword tokenization is a promising technique for symbolic music generation and may have broader implications for music composition.
  arXiv  Detail & Related papers  (2023-04-18T12:46:12Z)
• Byte Pair Encoding for Symbolic Music [0.0]
  Byte Pair embeddings significantly decreases the sequence length while increasing the vocabulary size. We leverage the embedding capabilities of such models with more expressive tokens, resulting in both better results and faster inference in generation and classification tasks. The source code is shared on Github, along with a companion website.
  arXiv  Detail & Related papers  (2023-01-27T20:22:18Z)
• Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation [138.74751744348274]
  We propose Museformer, a Transformer with a novel fine- and coarse-grained attention for music generation. Specifically, with the fine-grained attention, a token of a specific bar directly attends to all the tokens of the bars that are most relevant to music structures. With the coarse-grained attention, a token only attends to the summarization of the other bars rather than each token of them so as to reduce the computational cost.
  arXiv  Detail & Related papers  (2022-10-19T07:31:56Z)
• Quantized GAN for Complex Music Generation from Dance Videos [48.196705493763986]
  We present Dance2Music-GAN (D2M-GAN), a novel adversarial multi-modal framework that generates musical samples conditioned on dance videos. Our proposed framework takes dance video frames and human body motion as input, and learns to generate music samples that plausibly accompany the corresponding input.
  arXiv  Detail & Related papers  (2022-04-01T17:53:39Z)
• Music Gesture for Visual Sound Separation [121.36275456396075]
  "Music Gesture" is a keypoint-based structured representation to explicitly model the body and finger movements of musicians when they perform music. We first adopt a context-aware graph network to integrate visual semantic context with body dynamics, and then apply an audio-visual fusion model to associate body movements with the corresponding audio signals.
  arXiv  Detail & Related papers  (2020-04-20T17:53:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.