Generating Music with Structure Using Self-Similarity as Attention

• URL: http://arxiv.org/abs/2406.15647v2
• Date: Tue, 25 Jun 2024 18:26:07 GMT
• Title: Generating Music with Structure Using Self-Similarity as Attention
• Authors: Sophia Hager, Kathleen Hablutzel, Katherine M. Kinnaird,
• Abstract summary: We propose an attention layer that uses a novel approach applying user-supplied self-similarity to previous time steps. We demonstrate it in our Similarity Incentivized Generator (SING) system, a deep learning autonomous music generation system. We train SING on the MAESTRO dataset using a novel variable method, and compare its performance to the same model without the attention mechanism.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the innovations in deep learning and generative AI, creating long term structure as well as the layers of repeated structure common in musical works remains an open challenge in music generation. We propose an attention layer that uses a novel approach applying user-supplied self-similarity matrices to previous time steps, and demonstrate it in our Similarity Incentivized Neural Generator (SING) system, a deep learning autonomous music generation system with two layers. The first is a vanilla Long Short Term Memory layer, and the second is the proposed attention layer. During generation, this attention mechanism imposes a suggested structure from a template piece on the generated music. We train SING on the MAESTRO dataset using a novel variable batching method, and compare its performance to the same model without the attention mechanism. The addition of our proposed attention mechanism significantly improves the network's ability to replicate specific structures, and it performs better on an unseen test set than a model without the attention mechanism.

Related papers

• A Primal-Dual Framework for Transformers and Neural Networks [52.814467832108875]
  Self-attention is key to the remarkable success of transformers in sequence modeling tasks. We show that the self-attention corresponds to the support vector expansion derived from a support vector regression problem. We propose two new attentions: Batch Normalized Attention (Attention-BN) and Attention with Scaled Head (Attention-SH)
  arXiv  Detail & Related papers  (2024-06-19T19:11:22Z)
• Hierarchical Recurrent Neural Networks for Conditional Melody Generation with Long-term Structure [0.0]
  We propose a conditional melody generation model based on a hierarchical recurrent neural network. This model generates melodies with long-term structures based on given chord accompaniments. Results from our listening test indicate that CM-HRNN outperforms AttentionRNN in terms of long-term structure and overall rating.
  arXiv  Detail & Related papers  (2021-02-19T08:22:26Z)
• A framework to compare music generative models using automatic evaluation metrics extended to rhythm [69.2737664640826]
  This paper takes the framework proposed in a previous research that did not consider rhythm to make a series of design decisions, then, rhythm support is added to evaluate the performance of two RNN memory cells in the creation of monophonic music. The model considers the handling of music transposition and the framework evaluates the quality of the generated pieces using automatic quantitative metrics based on geometry which have rhythm support added as well.
  arXiv  Detail & Related papers  (2021-01-19T15:04:46Z)
• Sequence Generation using Deep Recurrent Networks and Embeddings: A study case in music [69.2737664640826]
  This paper evaluates different types of memory mechanisms (memory cells) and analyses their performance in the field of music composition. A set of quantitative metrics is presented to evaluate the performance of the proposed architecture automatically.
  arXiv  Detail & Related papers  (2020-12-02T14:19:19Z)
• A Self-Supervised Gait Encoding Approach with Locality-Awareness for 3D Skeleton Based Person Re-Identification [65.18004601366066]
  Person re-identification (Re-ID) via gait features within 3D skeleton sequences is a newly-emerging topic with several advantages. This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn gait representations for person Re-ID.
  arXiv  Detail & Related papers  (2020-09-05T16:06:04Z)
• Unconditional Audio Generation with Generative Adversarial Networks and Cycle Regularization [48.55126268721948]
  We present a generative adversarial network (GAN)-based model for unconditional generation of the mel-spectrograms of singing voices. We employ a hierarchical architecture in the generator to induce some structure in the temporal dimension. We evaluate the performance of the new model not only for generating singing voices, but also for generating speech voices.
  arXiv  Detail & Related papers  (2020-05-18T08:35:16Z)
• Music Generation with Temporal Structure Augmentation [0.0]
  The proposed method augments a connectionist generation model with count-down to song conclusion and meter markers as extra input features. An RNN architecture with LSTM cells is trained on the Nottingham folk music dataset in a supervised sequence learning setup. Experiments show an improved prediction performance for both types of annotation.
  arXiv  Detail & Related papers  (2020-04-21T19:19:58Z)
• RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning [69.20460466735852]
  This paper presents a deep reinforcement learning algorithm for online accompaniment generation. The proposed algorithm is able to respond to the human part and generate a melodic, harmonic and diverse machine part.
  arXiv  Detail & Related papers  (2020-02-08T03:53:52Z)
• Attentional networks for music generation [5.012960295592238]
  We propose a deep learning based music generation method in order to produce old style music particularly JAZZ with rehashed melodic structures. Owing to the success in modelling long-term temporal dependencies in sequential data and its success in case of videos, Bi-LSTMs with attention serve as the natural choice and early utilization in music generation. We validate in our experiments that Bi-LSTMs with attention are able to preserve the richness and technical nuances of the music performed.
  arXiv  Detail & Related papers  (2020-02-06T13:26:17Z)
• Continuous Melody Generation via Disentangled Short-Term Representations and Structural Conditions [14.786601824794369]
  We present a model for composing melodies given a user specified symbolic scenario combined with a previous music context. Our model is capable of generating long melodies by regarding 8-beat note sequences as basic units, and shares consistent rhythm pattern structure with another specific song. Results show that the music generated by our model tends to have salient repetition structures, rich motives, and stable rhythm patterns.
  arXiv  Detail & Related papers  (2020-02-05T06:23:44Z)
• Modeling Musical Structure with Artificial Neural Networks [0.0]
  I explore the application of artificial neural networks to different aspects of musical structure modeling. I show how a connectionist model, the Gated Autoencoder (GAE), can be employed to learn transformations between musical fragments. I propose a special predictive training of the GAE, which yields a representation of polyphonic music as a sequence of intervals.
  arXiv  Detail & Related papers  (2020-01-06T18:35:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.