An Ensemble Approach to Music Source Separation: A Comparative Analysis of Conventional and Hierarchical Stem Separation

• URL: http://arxiv.org/abs/2410.20773v1
• Date: Mon, 28 Oct 2024 06:18:12 GMT
• Title: An Ensemble Approach to Music Source Separation: A Comparative Analysis of Conventional and Hierarchical Stem Separation
• Authors: Saarth Vardhan, Pavani R Acharya, Samarth S Rao, Oorjitha Ratna Jasthi, S Natarajan,
• Abstract summary: Music source separation (MSS) is a task that involves isolating individual sound sources, or stems, from mixed audio signals. This paper presents an ensemble approach to MSS, combining several state-of-the-art architectures to achieve superior separation performance.
• Score: 0.4893345190925179
• License:
• Abstract: Music source separation (MSS) is a task that involves isolating individual sound sources, or stems, from mixed audio signals. This paper presents an ensemble approach to MSS, combining several state-of-the-art architectures to achieve superior separation performance across traditional Vocal, Drum, and Bass (VDB) stems, as well as expanding into second-level hierarchical separation for sub-stems like kick, snare, lead vocals, and background vocals. Our method addresses the limitations of relying on a single model by utilising the complementary strengths of various models, leading to more balanced results across stems. For stem selection, we used the harmonic mean of Signal-to-Noise Ratio (SNR) and Signal-to-Distortion Ratio (SDR), ensuring that extreme values do not skew the results and that both metrics are weighted effectively. In addition to consistently high performance across the VDB stems, we also explored second-level hierarchical separation, revealing important insights into the complexities of MSS and how factors like genre and instrumentation can influence model performance. While the second-level separation results show room for improvement, the ability to isolate sub-stems marks a significant advancement. Our findings pave the way for further research in MSS, particularly in expanding model capabilities beyond VDB and improving niche stem separations such as guitar and piano.

Related papers

• A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems [53.30852012059025]
  Banquet is a system that allows source separation of multiple stems using just one decoder. A bandsplit source separation model is extended to work in a query-based setup in tandem with a music instrument recognition PaSST model.
  arXiv  Detail & Related papers  (2024-06-26T20:25:53Z)
• High-Quality Visually-Guided Sound Separation from Diverse Categories [56.92841782969847]
  DAVIS is a Diffusion-based Audio-VIsual Separation framework. It synthesizes separated sounds directly from Gaussian noise, conditioned on both the audio mixture and the visual information. We compare DAVIS to existing state-of-the-art discriminative audio-visual separation methods on the AVE and MUSIC datasets.
  arXiv  Detail & Related papers  (2023-07-31T19:41:49Z)
• On Neural Architectures for Deep Learning-based Source Separation of Co-Channel OFDM Signals [104.11663769306566]
  We study the single-channel source separation problem involving frequency-division multiplexing (OFDM) signals. We propose critical domain-informed modifications to the network parameterization, based on insights from OFDM structures.
  arXiv  Detail & Related papers  (2023-03-11T16:29:13Z)
• Multi-Dimensional and Multi-Scale Modeling for Speech Separation Optimized by Discriminative Learning [9.84949849886926]
  Intra-SE-Conformer and Inter-Transformer (ISCIT) for speech separation. New network SE-Conformer can model audio sequences in multiple dimensions and scales.
  arXiv  Detail & Related papers  (2023-03-07T08:53:20Z)
• Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation [23.758202121043805]
  We propose a novel network to unify speech enhancement and separation with gradient modulation to improve noise-robustness. Experimental results show that our approach achieves the state-of-the-art on large-scale Libri2Mix- and Libri3Mix-noisy datasets.
  arXiv  Detail & Related papers  (2023-02-22T03:54:50Z)
• Inference and Denoise: Causal Inference-based Neural Speech Enhancement [83.4641575757706]
  This study addresses the speech enhancement (SE) task within the causal inference paradigm by modeling the noise presence as an intervention. The proposed causal inference-based speech enhancement (CISE) separates clean and noisy frames in an intervened noisy speech using a noise detector and assigns both sets of frames to two mask-based enhancement modules (EMs) to perform noise-conditional SE.
  arXiv  Detail & Related papers  (2022-11-02T15:03:50Z)
• Music Source Separation with Band-split RNN [25.578400006180527]
  We propose a frequency-domain model that splits the spectrogram of the mixture into subbands and perform interleaved band-level and sequence-level modeling. The choices of the bandwidths of the subbands can be determined by a priori knowledge or expert knowledge on the characteristics of the target source. Experiment results show that BSRNN trained only on MUSDB18-HQ dataset significantly outperforms several top-ranking models in Music Demixing (MDX) Challenge 2021.
  arXiv  Detail & Related papers  (2022-09-30T01:49:52Z)
• Music Separation Enhancement with Generative Modeling [11.545349346125743]
  We propose a post-processing model (the Make it Sound Good) to enhance the output of music source separation systems. Crowdsourced subjective evaluations demonstrate that human listeners prefer source estimates of bass and drums that have been post-processed by MSG.
  arXiv  Detail & Related papers  (2022-08-26T00:44:37Z)
• Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis [96.46952672172021]
  Bi-Bimodal Fusion Network (BBFN) is a novel end-to-end network that performs fusion on pairwise modality representations. Model takes two bimodal pairs as input due to known information imbalance among modalities.
  arXiv  Detail & Related papers  (2021-07-28T23:33:42Z)
• Multitask learning for instrument activation aware music source separation [83.30944624666839]
  We propose a novel multitask structure to investigate using instrument activation information to improve source separation performance. We investigate our system on six independent instruments, a more realistic scenario than the three instruments included in the widely-used MUSDB dataset. The results show that our proposed multitask model outperforms the baseline Open-Unmix model on the mixture of Mixing Secrets and MedleyDB dataset.
  arXiv  Detail & Related papers  (2020-08-03T02:35:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.