Neural Multichannel Distant Speaker Diarization with Heavy-tailed Source Separation Model

• URL: http://arxiv.org/abs/2609.12154v1
• Date: Thu, 10 Sep 2026 19:41:03 GMT
• Title: Neural Multichannel Distant Speaker Diarization with Heavy-tailed Source Separation Model
• Abstract summary: This paper generalizes a neural model that jointly learns to perform blind source separation and diarization over speech mixtures.<n>Experiments show consistent large improvements in Diarization Error Rate (DER) and Jaccard Error Rate (JER) compared to the baseline on various corpora.
• Score: 12.362272115239522
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Distant speaker diarization remains challenging due to difficult acoustic environments, varying numbers of speakers and overlapping speech. Model-driven methods are proposed to exploit the speech source features in multi-channel recordings that help diarization. This paper generalizes a neural model that jointly learns to perform blind source separation and diarization over speech mixtures (neural FCASA) with heavy-tailed models. The popular Gaussian distribution has been applied for variance modeling in the original source separation model, which we replace with two families of heavy-tailed models (Leptokurtic Generalized Gaussian distribution and Student's t distribution) to better capture the heavy-tailedness in speech signals. Thanks to the Gaussian scale mixture model, we are able to unify the proposed method and the original one under the same form of learning objective. Our experiments show consistent large improvements in Diarization Error Rate (DER) and Jaccard Error Rate (JER) compared to the baseline on various corpora.

Related papers

• Speech Enhancement Based on Drifting Models [2.6958419576949146]
  We propose Speech Enhancement based on Drifting Models (DriftSE)<n>DriftSE is a novel generative framework that formulates denoising as an equilibrium problem.<n> Experiments on the VoiceBank-DEMAND demonstrate that DriftSE achieves high-fidelity enhancement in a single step.
  arXiv  Detail & Related papers  (2026-04-27T09:00:51Z)
• From Denoising to Refining: A Corrective Framework for Vision-Language Diffusion Model [72.73512218682187]
  We introduce ReDiff, a refining-enhanced diffusion framework that teaches the model to identify and correct its own errors.<n>Our approach features a two-stage training process: first, we instill a foundational revision capability by training the model to revise synthetic errors; second, we implement a novel online self-correction loop.<n>This mistake-driven learning endows the model with the crucial ability to revisit and refine its already generated output, effectively breaking the error cascade.
  arXiv  Detail & Related papers  (2025-10-22T06:58:55Z)
• Disentanglement of Sources in a Multi-Stream Variational Autoencoder [4.562056072136493]
  Variational autoencoders (VAEs) are a leading approach to address the problem of learning disentangled representations.<n>Here we explore a different approach by using discrete latents to combine VAE-representations of individual sources.
  arXiv  Detail & Related papers  (2025-10-17T13:54:56Z)
• Generalized Interpolating Discrete Diffusion [65.74168524007484]
  Masked diffusion is a popular choice due to its simplicity and effectiveness.<n>We generalize a new family of general interpolating discrete diffusion (GIDD) which offers greater flexibility in the design of the noising processes.<n>Exploiting GIDD's flexibility, we explore a hybrid approach combining masking and uniform noise, leading to improved sample quality.
  arXiv  Detail & Related papers  (2025-03-06T14:30:55Z)
• BEND: Bagging Deep Learning Training Based on Efficient Neural Network Diffusion [56.9358325168226]
  We propose a Bagging deep learning training algorithm based on Efficient Neural network Diffusion (BEND) Our approach is simple but effective, first using multiple trained model weights and biases as inputs to train autoencoder and latent diffusion model. Our proposed BEND algorithm can consistently outperform the mean and median accuracies of both the original trained model and the diffused model.
  arXiv  Detail & Related papers  (2024-03-23T08:40:38Z)
• One More Step: A Versatile Plug-and-Play Module for Rectifying Diffusion Schedule Flaws and Enhancing Low-Frequency Controls [77.42510898755037]
  One More Step (OMS) is a compact network that incorporates an additional simple yet effective step during inference. OMS elevates image fidelity and harmonizes the dichotomy between training and inference, while preserving original model parameters. Once trained, various pre-trained diffusion models with the same latent domain can share the same OMS module.
  arXiv  Detail & Related papers  (2023-11-27T12:02:42Z)
• Multi-scale Diffusion Denoised Smoothing [79.95360025953931]
  randomized smoothing has become one of a few tangible approaches that offers adversarial robustness to models at scale. We present scalable methods to address the current trade-off between certified robustness and accuracy in denoised smoothing. Our experiments show that the proposed multi-scale smoothing scheme combined with diffusion fine-tuning enables strong certified robustness available with high noise level.
  arXiv  Detail & Related papers  (2023-10-25T17:11:21Z)
• Denoising Diffusion Bridge Models [54.87947768074036]
  Diffusion models are powerful generative models that map noise to data using processes. For many applications such as image editing, the model input comes from a distribution that is not random noise. In our work, we propose Denoising Diffusion Bridge Models (DDBMs)
  arXiv  Detail & Related papers  (2023-09-29T03:24:24Z)
• Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separation [99.19786288094596]
  We show how the upper bound can be generalized to the case of random generative models. We show state-of-the-art results on 2, 3, 5, 10, and 20 speakers on multiple benchmarks.
  arXiv  Detail & Related papers  (2023-01-25T18:21:51Z)
• A weighted-variance variational autoencoder model for speech enhancement [0.0]
  We propose a weighted variance generative model, where the contribution of each spectrogram time-frame in parameter learning is weighted. We develop efficient training and speech enhancement algorithms based on the proposed generative model.
  arXiv  Detail & Related papers  (2022-11-02T09:51:15Z)
• Speech Prediction in Silent Videos using Variational Autoencoders [29.423462898526605]
  We present a model for generating speech in a silent video. The proposed model combines recurrent neural networks and variational deep generative models to learn the auditory's conditional distribution. We demonstrate the performance of our model on the GRID dataset based on standard benchmarks.
  arXiv  Detail & Related papers  (2020-11-14T17:09:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.