Source Separation of Multi-source Raw Music using a Residual Quantized Variational Autoencoder

• URL: http://arxiv.org/abs/2408.07020v1
• Date: Mon, 12 Aug 2024 17:30:17 GMT
• Title: Source Separation of Multi-source Raw Music using a Residual Quantized Variational Autoencoder
• Authors: Leonardo Berti,
• Abstract summary: I develop a neural audio model based on the residual quantized variational autoencoder architecture. The model can separate audio sources, achieving almost SoTA results with much less computing power.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: I developed a neural audio codec model based on the residual quantized variational autoencoder architecture. I train the model on the Slakh2100 dataset, a standard dataset for musical source separation, composed of multi-track audio. The model can separate audio sources, achieving almost SoTA results with much less computing power. The code is publicly available at github.com/LeonardoBerti00/Source-Separation-of-Multi-source-Music-using-Residual-Quantizad-Variatio nal-Autoencoder

Related papers

• Learning Source Disentanglement in Neural Audio Codec [20.335701584949526]
  We introduce the Source-Disentangled Neural Audio Codec (SD-Codec), a novel approach that combines audio coding and source separation. By jointly learning audio resynthesis and separation, SD-Codec explicitly assigns audio signals from different domains to distinct codebooks, sets of discrete representations. Experimental results indicate that SD-Codec not only maintains competitive resynthesis quality but also, supported by the separation results, demonstrates successful disentanglement of different sources in the latent space.
  arXiv  Detail & Related papers  (2024-09-17T14:21:02Z)
• Multi-Source Music Generation with Latent Diffusion [7.832209959041259]
  Multi-Source Diffusion Model (MSDM) proposed to model music as a mixture of multiple instrumental sources. MSLDM employs Variational Autoencoders (VAEs) to encode each instrumental source into a distinct latent representation. This approach significantly enhances the total and partial generation of music.
  arXiv  Detail & Related papers  (2024-09-10T03:41:10Z)
• WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling [65.30937248905958]
  A crucial component of language models is the tokenizer, which compresses high-dimensional natural signals into lower-dimensional discrete tokens. We introduce WavTokenizer, which offers several advantages over previous SOTA acoustic models in the audio domain. WavTokenizer achieves state-of-the-art reconstruction quality with outstanding UTMOS scores and inherently contains richer semantic information.
  arXiv  Detail & Related papers  (2024-08-29T13:43:36Z)
• An Independence-promoting Loss for Music Generation with Language Models [64.95095558672996]
  Music generation schemes rely on a vocabulary of audio tokens, generally provided as codes in a discrete latent space learnt by an auto-encoder. We introduce an independence-promoting loss to regularize the auto-encoder used as the tokenizer in language models for music generation.
  arXiv  Detail & Related papers  (2024-06-04T13:44:39Z)
• Separate Anything You Describe [55.0784713558149]
  Language-queried audio source separation (LASS) is a new paradigm for computational auditory scene analysis (CASA) AudioSep is a foundation model for open-domain audio source separation with natural language queries.
  arXiv  Detail & Related papers  (2023-08-09T16:09:44Z)
• Benchmarks and leaderboards for sound demixing tasks [44.99833362998488]
  We introduce two new benchmarks for the sound source separation tasks. We compare popular models for sound demixing, as well as their ensembles, on these benchmarks. We also develop a novel approach for audio separation, based on the ensembling of different models that are suited best for the particular stem.
  arXiv  Detail & Related papers  (2023-05-12T14:00:26Z)
• Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models [65.18102159618631]
  multimodal generative modeling has created milestones in text-to-image and text-to-video generation. Its application to audio still lags behind for two main reasons: the lack of large-scale datasets with high-quality text-audio pairs, and the complexity of modeling long continuous audio data. We propose Make-An-Audio with a prompt-enhanced diffusion model that addresses these gaps.
  arXiv  Detail & Related papers  (2023-01-30T04:44:34Z)
• High Fidelity Neural Audio Compression [92.4812002532009]
  We introduce a state-of-the-art real-time, high-fidelity, audio leveraging neural networks. It consists in a streaming encoder-decoder architecture with quantized latent space trained in an end-to-end fashion. We simplify and speed-up the training by using a single multiscale spectrogram adversary.
  arXiv  Detail & Related papers  (2022-10-24T17:52:02Z)
• AudioGen: Textually Guided Audio Generation [116.57006301417306]
  We tackle the problem of generating audio samples conditioned on descriptive text captions. In this work, we propose AaudioGen, an auto-regressive model that generates audio samples conditioned on text inputs.
  arXiv  Detail & Related papers  (2022-09-30T10:17:05Z)
• Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data [26.058278155958668]
  We propose a three-component pipeline to train a universal audio source separator from a large, but weakly-labeled dataset: AudioSet. Our approach uses a single model for source separation of multiple sound types, and relies solely on weakly-labeled data for training. The proposed audio separator can be used in a zero-shot setting, learning to separate types of audio sources that were never seen in training.
  arXiv  Detail & Related papers  (2021-12-15T05:13:43Z)
• Unsupervised Source Separation By Steering Pretrained Music Models [15.847814664948013]
  We showcase an unsupervised method that repurposes deep models trained for music generation and music tagging for audio source separation. An audio generation model is conditioned on an input mixture, producing a latent encoding of the audio used to generate audio. This generated audio is fed to a pretrained music tagger that creates source labels.
  arXiv  Detail & Related papers  (2021-10-25T16:08:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.