Benchmarks and leaderboards for sound demixing tasks

• URL: http://arxiv.org/abs/2305.07489v2
• Date: Tue, 7 May 2024 10:35:10 GMT
• Title: Benchmarks and leaderboards for sound demixing tasks
• Authors: Roman Solovyev, Alexander Stempkovskiy, Tatiana Habruseva,
• Abstract summary: We introduce two new benchmarks for the sound source separation tasks. We compare popular models for sound demixing, as well as their ensembles, on these benchmarks. We also develop a novel approach for audio separation, based on the ensembling of different models that are suited best for the particular stem.
• Score: 44.99833362998488
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Music demixing is the task of separating different tracks from the given single audio signal into components, such as drums, bass, and vocals from the rest of the accompaniment. Separation of sources is useful for a range of areas, including entertainment and hearing aids. In this paper, we introduce two new benchmarks for the sound source separation tasks and compare popular models for sound demixing, as well as their ensembles, on these benchmarks. For the models' assessments, we provide the leaderboard at https://mvsep.com/quality_checker/, giving a comparison for a range of models. The new benchmark datasets are available for download. We also develop a novel approach for audio separation, based on the ensembling of different models that are suited best for the particular stem. The proposed solution was evaluated in the context of the Music Demixing Challenge 2023 and achieved top results in different tracks of the challenge. The code and the approach are open-sourced on GitHub.

Related papers

• Separate This, and All of these Things Around It: Music Source Separation via Hyperellipsoidal Queries [53.30852012059025]
  Music source separation is an audio-to-audio retrieval task. Recent work in music source separation has begun to challenge the fixed-stem paradigm. We propose the use of hyperellipsoidal regions as queries to allow for an intuitive yet easily parametrizable approach to specifying both the target (location) and its spread.
  arXiv  Detail & Related papers  (2025-01-27T16:13:50Z)
• Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis [56.01110988816489]
  We propose to synthesize high-quality and synchronized audio, given video and optional text conditions, using a novel multimodal joint training framework MMAudio. MMAudio is jointly trained with larger-scale, readily available text-audio data to learn to generate semantically aligned high-quality audio samples. MMAudio achieves surprisingly competitive performance in text-to-audio generation, showing that joint training does not hinder single-modality performance.
  arXiv  Detail & Related papers  (2024-12-19T18:59:55Z)
• Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models [56.776580717999806]
  Real-world applications often involve processing multiple audio streams simultaneously. We propose the first multi-audio evaluation benchmark that consists of 20 datasets from 11 multi-audio tasks. We propose a novel multi-audio-LLM (MALLM) to capture audio context among multiple similar audios.
  arXiv  Detail & Related papers  (2024-09-27T12:06:53Z)
• Semantic Grouping Network for Audio Source Separation [41.54814517077309]
  We present a novel Semantic Grouping Network, termed as SGN, that can directly disentangle sound representations and extract high-level semantic information for each source from input audio mixture. We conducted extensive experiments on music-only and universal sound separation benchmarks: MUSIC, FUSS, MUSDB18, and VGG-Sound.
  arXiv  Detail & Related papers  (2024-07-04T08:37:47Z)
• AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codes [6.375996974877916]
  We propose a method named AudioFormer, which learns audio feature representations through the acquisition of discrete acoustic codes. Our research outcomes demonstrate that AudioFormer attains significantly improved performance compared to prevailing monomodal audio classification models.
  arXiv  Detail & Related papers  (2023-08-14T15:47:25Z)
• Separate Anything You Describe [53.30484933564858]
  Language-queried audio source separation (LASS) is a new paradigm for computational auditory scene analysis (CASA) AudioSep is a foundation model for open-domain audio source separation with natural language queries.
  arXiv  Detail & Related papers  (2023-08-09T16:09:44Z)
• AudioGen: Textually Guided Audio Generation [116.57006301417306]
  We tackle the problem of generating audio samples conditioned on descriptive text captions. In this work, we propose AaudioGen, an auto-regressive model that generates audio samples conditioned on text inputs.
  arXiv  Detail & Related papers  (2022-09-30T10:17:05Z)
• Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data [26.058278155958668]
  We propose a three-component pipeline to train a universal audio source separator from a large, but weakly-labeled dataset: AudioSet. Our approach uses a single model for source separation of multiple sound types, and relies solely on weakly-labeled data for training. The proposed audio separator can be used in a zero-shot setting, learning to separate types of audio sources that were never seen in training.
  arXiv  Detail & Related papers  (2021-12-15T05:13:43Z)
• Modeling the Compatibility of Stem Tracks to Generate Music Mashups [6.922825755771942]
  A music mashup combines audio elements from two or more songs to create a new work. Research has developed algorithms that predict the compatibility of audio elements.
  arXiv  Detail & Related papers  (2021-03-26T01:51:11Z)
• Leveraging Category Information for Single-Frame Visual Sound Source Separation [15.26733033527393]
  We study simple yet efficient models for visual sound separation using only a single video frame. Our models are able to exploit the information of the sound source category in the separation process.
  arXiv  Detail & Related papers  (2020-07-15T20:35:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.