Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation

• URL: http://arxiv.org/abs/2408.16126v1
• Date: Wed, 28 Aug 2024 20:26:34 GMT
• Title: Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation
• Authors: Ke Chen, Jiaqi Su, Taylor Berg-Kirkpatrick, Shlomo Dubnov, Zeyu Jin,
• Abstract summary: We present a novel data simulation pipeline that produces diverse training data from a range of acoustic environments and content. We propose new training paradigms to improve quality of a general speech separation model.
• Score: 45.76004686788507
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Achieving robust speech separation for overlapping speakers in various acoustic environments with noise and reverberation remains an open challenge. Although existing datasets are available to train separators for specific scenarios, they do not effectively generalize across diverse real-world scenarios. In this paper, we present a novel data simulation pipeline that produces diverse training data from a range of acoustic environments and content, and propose new training paradigms to improve quality of a general speech separation model. Specifically, we first introduce AC-SIM, a data simulation pipeline that incorporates broad variations in both content and acoustics. Then we integrate multiple training objectives into the permutation invariant training (PIT) to enhance separation quality and generalization of the trained model. Finally, we conduct comprehensive objective and human listening experiments across separation architectures and benchmarks to validate our methods, demonstrating substantial improvement of generalization on both non-homologous and real-world test sets.

Related papers

• Developing an Effective Training Dataset to Enhance the Performance of AI-based Speaker Separation Systems [0.3277163122167434]
  We propose a novel method for constructing a realistic training set that includes mixture signals and corresponding ground truths for each speaker. We get a 1.65 dB improvement in Scale Invariant Signal to Distortion Ratio (SI-SDR) for speaker separation accuracy in realistic mixing.
  arXiv  Detail & Related papers  (2024-11-13T06:55:18Z)
• UMSE: Unified Multi-scenario Summarization Evaluation [52.60867881867428]
  Summarization quality evaluation is a non-trivial task in text summarization. We propose Unified Multi-scenario Summarization Evaluation Model (UMSE) Our UMSE is the first unified summarization evaluation framework engaged with the ability to be used in three evaluation scenarios.
  arXiv  Detail & Related papers  (2023-05-26T12:54:44Z)
• Multi-source Domain Adaptation for Text-independent Forensic Speaker Recognition [36.83842373791537]
  Adapting speaker recognition systems to new environments is a widely-used technique to improve a well-performing model. Previous studies focus on single domain adaptation, which neglects a more practical scenario where training data are collected from multiple acoustic domains. Three novel adaptation methods are proposed to further promote adaptation performance across multiple acoustic domains.
  arXiv  Detail & Related papers  (2022-11-17T22:11:25Z)
• Adaptive Multi-Corpora Language Model Training for Speech Recognition [13.067901680326932]
  We introduce a novel adaptive multi-corpora training algorithm that dynamically learns and adjusts the sampling probability of each corpus along the training process. Compared with static sampling strategy baselines, the proposed approach yields remarkable improvement.
  arXiv  Detail & Related papers  (2022-11-09T06:54:50Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Heterogeneous Target Speech Separation [52.05046029743995]
  We introduce a new paradigm for single-channel target source separation where the sources of interest can be distinguished using non-mutually exclusive concepts. Our proposed heterogeneous separation framework can seamlessly leverage datasets with large distribution shifts.
  arXiv  Detail & Related papers  (2022-04-07T17:14:20Z)
• MetaAudio: A Few-Shot Audio Classification Benchmark [2.294014185517203]
  This work aims to alleviate this reliance on image-based benchmarks by offering the first comprehensive, public and fully reproducible audio based alternative. We compare the few-shot classification performance of a variety of techniques on seven audio datasets. Our experimentation shows gradient-based meta-learning methods such as MAML and Meta-Curvature consistently outperform both metric and baseline methods.
  arXiv  Detail & Related papers  (2022-04-05T11:33:44Z)
• An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition [98.70304981174748]
  We focus on the general applications of pretrained speech representations, on advanced end-to-end automatic speech recognition (E2E-ASR) models. We select several pretrained speech representations and present the experimental results on various open-source and publicly available corpora for E2E-ASR.
  arXiv  Detail & Related papers  (2021-10-09T15:06:09Z)
• End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddings [66.50782702086575]
  We present an end-to-end deep network model that performs meeting diarization from single-channel audio recordings. The proposed system is designed to handle meetings with unknown numbers of speakers, using variable-number permutation-invariant cross-entropy based loss functions.
  arXiv  Detail & Related papers  (2021-05-05T14:55:29Z)
• Improving speech recognition models with small samples for air traffic control systems [9.322392779428505]
  In this work, a novel training approach based on pretraining and transfer learning is proposed to address the issue of small training samples. Three real ATC datasets are used to validate the proposed ASR model and training strategies. The experimental results demonstrate that the ASR performance is significantly improved on all three datasets.
  arXiv  Detail & Related papers  (2021-02-16T08:28:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.