MelodyVis: Visual Analytics for Melodic Patterns in Sheet Music

• URL: http://arxiv.org/abs/2407.05427v1
• Date: Sun, 07 Jul 2024 16:04:21 GMT
• Title: MelodyVis: Visual Analytics for Melodic Patterns in Sheet Music
• Abstract summary: MelodyVis is a visual application designed in collaboration with musicology experts to explore melodic patterns in digital sheet music. The system utilizes eight atomic operators, such as transposition and mirroring, to capture melody repetitions and variations.
• Score: 7.712907363185298
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Manual melody detection is a tedious task requiring high expertise level, while automatic detection is often not expressive or powerful enough. Thus, we present MelodyVis, a visual application designed in collaboration with musicology experts to explore melodic patterns in digital sheet music. MelodyVis features five connected views, including a Melody Operator Graph and a Voicing Timeline. The system utilizes eight atomic operators, such as transposition and mirroring, to capture melody repetitions and variations. Users can start their analysis by manually selecting patterns in the sheet view, and then identifying other patterns based on the selected samples through an interactive exploration process. We conducted a user study to investigate the effectiveness and usefulness of our approach and its integrated melodic operators, including usability and mental load questions. We compared the analysis executed by 25 participants with and without the operators. The study results indicate that the participants could identify at least twice as many patterns with activated operators. MelodyVis allows analysts to steer the analysis process and interpret results. Our study also confirms the usefulness of MelodyVis in supporting common analytical tasks in melodic analysis, with participants reporting improved pattern identification and interpretation. Thus, MelodyVis addresses the limitations of fully-automated approaches, enabling music analysts to step into the analysis process and uncover and understand intricate melodic patterns and transformations in sheet music.

Related papers

• Discovering "Words" in Music: Unsupervised Learning of Compositional Sparse Code for Symbolic Music [50.87225308217594]
  This paper presents an unsupervised machine learning algorithm that identifies recurring patterns -- referred to as music-words'' -- from symbolic music data.<n>We formulate the task of music-word discovery as a statistical optimization problem and propose a two-stage Expectation-Maximization (EM)-based learning framework.
  arXiv  Detail & Related papers  (2025-09-29T11:10:57Z)
• Towards an AI Musician: Synthesizing Sheet Music Problems for Musical Reasoning [69.78158549955384]
  We introduce a novel approach that treats core music theory rules, such as those governing beats and intervals, as programmatic functions.<n>This approach generates verifiable sheet music questions in both textual and visual modalities.<n> Evaluation results on SSMR-Bench highlight the key role reasoning plays in interpreting sheet music.
  arXiv  Detail & Related papers  (2025-09-04T09:42:17Z)
• Evaluating Interval-based Tokenization for Pitch Representation in Symbolic Music Analysis [0.10241134756773229]
  We introduce a general framework for building interval-based tokenizations. We show that interval-based tokenizations improve model performances and facilitate their explainability.
  arXiv  Detail & Related papers  (2025-01-08T17:22:03Z)
• Automatic Estimation of Singing Voice Musical Dynamics [9.343063100314687]
  We propose a methodology for dataset curation. We compile a dataset comprising 509 musical dynamics annotated singing voice performances, aligned with 163 score files. We train a CNN model with varying window sizes to evaluate the effectiveness of estimating musical dynamics. We conclude through our experiments that bark-scale based features outperform log-Mel-features for the task of singing voice dynamics prediction.
  arXiv  Detail & Related papers  (2024-10-27T18:15:18Z)
• What User Behaviors Make the Differences During the Process of Visual Analytics? [1.5285292154680246]
  This work presents a study on a comprehensive data collection of user behaviors, and our analysis approach with time-series classification methods. Our user study collects user behaviors on a diverse set of visualization tasks with two comparable systems, desktop and immersive visualizations. Our results reveal that user behaviors can be distinguished during the process of visual analytics and there is a potentially strong association between the physical behaviors of users and the visualization tasks they perform.
  arXiv  Detail & Related papers  (2023-11-01T17:45:52Z)
• Motif-Centric Representation Learning for Symbolic Music [5.781931021964343]
  We learn the implicit relationship between motifs and their variations via representation learning. A regularization-based method, VICReg, is adopted for pretraining, while contrastive learning is used for fine-tuning. We visualize the acquired motif representations, offering an intuitive comprehension of the overall structure of a music piece.
  arXiv  Detail & Related papers  (2023-09-19T13:09:03Z)
• Melody transcription via generative pre-training [86.08508957229348]
  Key challenge in melody transcription is building methods which can handle broad audio containing any number of instrument ensembles and musical styles. To confront this challenge, we leverage representations from Jukebox (Dhariwal et al. 2020), a generative model of broad music audio. We derive a new dataset containing $50$ hours of melody transcriptions from crowdsourced annotations of broad music.
  arXiv  Detail & Related papers  (2022-12-04T18:09:23Z)
• Exploring the Efficacy of Pre-trained Checkpoints in Text-to-Music Generation Task [86.72661027591394]
  We generate complete and semantically consistent symbolic music scores from text descriptions. We explore the efficacy of using publicly available checkpoints for natural language processing in the task of text-to-music generation. Our experimental results show that the improvement from using pre-trained checkpoints is statistically significant in terms of BLEU score and edit distance similarity.
  arXiv  Detail & Related papers  (2022-11-21T07:19:17Z)
• A Novel Multi-Task Learning Method for Symbolic Music Emotion Recognition [76.65908232134203]
  Symbolic Music Emotion Recognition(SMER) is to predict music emotion from symbolic data, such as MIDI and MusicXML. In this paper, we present a simple multi-task framework for SMER, which incorporates the emotion recognition task with other emotion-related auxiliary tasks.
  arXiv  Detail & Related papers  (2022-01-15T07:45:10Z)
• Detecting Generic Music Features with Single Layer Feedforward Network using Unsupervised Hebbian Computation [3.8707695363745223]
  The authors extract information on such features from a popular open-source music corpus. They apply unsupervised Hebbian learning techniques on their single-layer neural network using the same dataset. The unsupervised training algorithm enhances their proposed neural network to achieve an accuracy of 90.36% for successful music feature detection.
  arXiv  Detail & Related papers  (2020-08-31T13:57:31Z)
• Visual Attention for Musical Instrument Recognition [72.05116221011949]
  We explore the use of attention mechanism in a timbral-temporal sense, a la visual attention, to improve the performance of musical instrument recognition. The first approach applies attention mechanism to the sliding-window paradigm, where a prediction based on each timbral-temporal instance' is given an attention weight, before aggregation to produce the final prediction. The second approach is based on a recurrent model of visual attention where the network only attends to parts of the spectrogram and decide where to attend to next, given a limited number of glimpses'
  arXiv  Detail & Related papers  (2020-06-17T03:56:44Z)
• Music Gesture for Visual Sound Separation [121.36275456396075]
  "Music Gesture" is a keypoint-based structured representation to explicitly model the body and finger movements of musicians when they perform music. We first adopt a context-aware graph network to integrate visual semantic context with body dynamics, and then apply an audio-visual fusion model to associate body movements with the corresponding audio signals.
  arXiv  Detail & Related papers  (2020-04-20T17:53:46Z)
• Exploring Inherent Properties of the Monophonic Melody of Songs [10.055143995729415]
  We propose a set of interpretable features on monophonic melody for computational purposes. These features are defined not only in mathematical form, but also with some considerations on composers 'intuition. These features are considered by people universally in many genres of songs, even for atonal composition practices.
  arXiv  Detail & Related papers  (2020-03-20T14:13:16Z)
• Multi-Modal Music Information Retrieval: Augmenting Audio-Analysis with Visual Computing for Improved Music Video Analysis [91.3755431537592]
  This thesis combines audio-analysis with computer vision to approach Music Information Retrieval (MIR) tasks from a multi-modal perspective. The main hypothesis of this work is based on the observation that certain expressive categories such as genre or theme can be recognized on the basis of the visual content alone. The experiments are conducted for three MIR tasks Artist Identification, Music Genre Classification and Cross-Genre Classification.
  arXiv  Detail & Related papers  (2020-02-01T17:57:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.