Exploring how a Generative AI interprets music
- URL: http://arxiv.org/abs/2308.00015v1
- Date: Mon, 31 Jul 2023 15:35:32 GMT
- Title: Exploring how a Generative AI interprets music
- Authors: Gabriela Barenboim, Luigi Del Debbio, Johannes Hirn, Veronica Sanz
- Abstract summary: We use Google's MusicVAE, a Variational Auto-Encoder with a 512-dimensional latent space to represent a few bars of music.
We find that, on average, most latent neurons remain silent when fed real music tracks.
The concept of melody only seems to show up in independent neurons for longer sequences of music.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: We use Google's MusicVAE, a Variational Auto-Encoder with a 512-dimensional
latent space to represent a few bars of music, and organize the latent
dimensions according to their relevance in describing music. We find that, on
average, most latent neurons remain silent when fed real music tracks: we call
these "noise" neurons. The remaining few dozens of latent neurons that do fire
are called "music neurons". We ask which neurons carry the musical information
and what kind of musical information they encode, namely something that can be
identified as pitch, rhythm or melody. We find that most of the information
about pitch and rhythm is encoded in the first few music neurons: the neural
network has thus constructed a couple of variables that non-linearly encode
many human-defined variables used to describe pitch and rhythm. The concept of
melody only seems to show up in independent neurons for longer sequences of
music.
Related papers
- Investigating the Encoding of Words in BERT's Neurons using Feature
Textualization [11.943486282441143]
We propose a technique to produce representations of neurons in embedding word space.
We find that the produced representations can provide insights about the encoded knowledge in individual neurons.
arXiv Detail & Related papers (2023-11-14T15:21:49Z) - Brain2Music: Reconstructing Music from Human Brain Activity [1.4777718769290527]
We introduce a method for reconstructing music from brain activity, captured using functional magnetic resonance imaging (fMRI)
Our approach uses either music retrieval or the MusicLM music generation model conditioned on embeddings derived from fMRI data.
The generated music resembles the musical stimuli that human subjects experienced, with respect to semantic properties like genre, instrumentation, and mood.
arXiv Detail & Related papers (2023-07-20T17:55:17Z) - Constraints on the design of neuromorphic circuits set by the properties
of neural population codes [61.15277741147157]
In the brain, information is encoded, transmitted and used to inform behaviour.
Neuromorphic circuits need to encode information in a way compatible to that used by populations of neuron in the brain.
arXiv Detail & Related papers (2022-12-08T15:16:04Z) - Natural Language Descriptions of Deep Visual Features [50.270035018478666]
We introduce a procedure that automatically labels neurons with open-ended, compositional, natural language descriptions.
We use MILAN for analysis, characterizing the distribution and importance of neurons selective for attribute, category, and relational information in vision models.
We also use MILAN for auditing, surfacing neurons sensitive to protected categories like race and gender in models trained on datasets intended to obscure these features.
arXiv Detail & Related papers (2022-01-26T18:48:02Z) - Comparing the Accuracy of Deep Neural Networks (DNN) and Convolutional
Neural Network (CNN) in Music Genre Recognition (MGR): Experiments on Kurdish
Music [0.0]
We developed a dataset that contains 880 samples from eight different Kurdish music genres.
We evaluated two machine learning approaches, a Deep Neural Network (DNN) and a Convolutional Neural Network (CNN) to recognize the genres.
arXiv Detail & Related papers (2021-11-22T09:21:48Z) - Astrocytes mediate analogous memory in a multi-layer neuron-astrocytic
network [52.77024349608834]
We show how a piece of information can be maintained as a robust activity pattern for several seconds then completely disappear if no other stimuli come.
This kind of short-term memory can keep operative information for seconds, then completely forget it to avoid overlapping with forthcoming patterns.
We show how arbitrary patterns can be loaded, then stored for a certain interval of time, and retrieved if the appropriate clue pattern is applied to the input.
arXiv Detail & Related papers (2021-08-31T16:13:15Z) - Multi-Channel Automatic Music Transcription Using Tensor Algebra [0.0]
This report aims at developing some of the existing techniques towards Music Transcription.
It will also introduce the concept of multi-channel automatic music transcription.
arXiv Detail & Related papers (2021-07-23T14:07:40Z) - MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training [97.91071692716406]
Symbolic music understanding refers to the understanding of music from the symbolic data.
MusicBERT is a large-scale pre-trained model for music understanding.
arXiv Detail & Related papers (2021-06-10T10:13:05Z) - Compositional Explanations of Neurons [52.71742655312625]
We describe a procedure for explaining neurons in deep representations by identifying compositional logical concepts.
We use this procedure to answer several questions on interpretability in models for vision and natural language processing.
arXiv Detail & Related papers (2020-06-24T20:37:05Z) - Non-linear Neurons with Human-like Apical Dendrite Activations [81.18416067005538]
We show that a standard neuron followed by our novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy.
We conduct experiments on six benchmark data sets from computer vision, signal processing and natural language processing.
arXiv Detail & Related papers (2020-02-02T21:09:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.