Multi-microphone Automatic Speech Segmentation in Meetings Based on Circular Harmonics Features

• URL: http://arxiv.org/abs/2306.04268v1
• Date: Wed, 7 Jun 2023 09:09:00 GMT
• Title: Multi-microphone Automatic Speech Segmentation in Meetings Based on Circular Harmonics Features
• Authors: Th\'eo Mariotte (LAUM, LIUM), Anthony Larcher (LIUM), Silvio Montr\'esor (LAUM), Jean-Hugh Thomas (LAUM)
• Abstract summary: We propose a new set of spatial features based on direction-of-arrival estimations in the circular harmonic domain (CH-DOA) Experiments on the AMI meeting corpus show that CH-DOA can improve the segmentation while being robust in the case of deactivated microphones.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speaker diarization is the task of answering Who spoke and when? in an audio stream. Pipeline systems rely on speech segmentation to extract speakers' segments and achieve robust speaker diarization. This paper proposes a common framework to solve three segmentation tasks in the distant speech scenario: Voice Activity Detection (VAD), Overlapped Speech Detection (OSD), and Speaker Change Detection (SCD). In the literature, a few studies investigate the multi-microphone distant speech scenario. In this work, we propose a new set of spatial features based on direction-of-arrival estimations in the circular harmonic domain (CH-DOA). These spatial features are extracted from multi-microphone audio data and combined with standard acoustic features. Experiments on the AMI meeting corpus show that CH-DOA can improve the segmentation while being robust in the case of deactivated microphones.

Related papers

• DiscreteSLU: A Large Language Model with Self-Supervised Discrete Speech Units for Spoken Language Understanding [51.32965203977845]
  We propose the use of discrete speech units (DSU) instead of continuous-valued speech encoder outputs. The proposed model shows robust performance on speech inputs from seen/unseen domains and instruction-following capability in spoken question answering. Our findings suggest that the ASR task and datasets are not crucial in instruction-tuning for spoken question answering tasks.
  arXiv  Detail & Related papers  (2024-06-13T17:28:13Z)
• ASoBO: Attentive Beamformer Selection for Distant Speaker Diarization in Meetings [4.125756306660331]
  Speaker Diarization (SD) aims at grouping speech segments that belong to the same speaker. Beamforming, i.e., spatial filtering, is a common practice to process multi-microphone audio data. This paper proposes a self-attention-based algorithm to select the output of a bank of fixed spatial filters.
  arXiv  Detail & Related papers  (2024-06-05T13:28:28Z)
• Online speaker diarization of meetings guided by speech separation [0.0]
  Overlapped speech is notoriously problematic for speaker diarization systems. We introduce a new speech separation-guided diarization scheme suitable for the online speaker diarization of long meeting recordings.
  arXiv  Detail & Related papers  (2024-01-30T09:09:22Z)
• Joint speech and overlap detection: a benchmark over multiple audio setup and speech domains [0.0]
  VAD and OSD can be trained jointly using a multi-class classification model. This paper proposes a complete and new benchmark of different VAD and OSD models. Our 2/3-class systems, which combine a Temporal Convolutional Network with speech representations adapted to the setup, outperform state-of-the-art results.
  arXiv  Detail & Related papers  (2023-07-24T14:29:21Z)
• Bi-LSTM Scoring Based Similarity Measurement with Agglomerative Hierarchical Clustering (AHC) for Speaker Diarization [0.0]
  A typical conversation between two speakers consists of segments where their voices overlap, interrupt each other or halt their speech in between multiple sentences. Recent advancements in diarization technology leverage neural network-based approaches to improvise speaker diarization system. We propose a Bi-directional Long Short-term Memory network for estimating the elements present in the similarity matrix.
  arXiv  Detail & Related papers  (2022-05-19T17:20:51Z)
• Continuous Speech Separation with Ad Hoc Microphone Arrays [35.87274524040486]
  Speech separation has been shown effective for multi-talker speech recognition. In this paper, we extend this approach to continuous speech separation. Two methods are proposed to mitigate a speech problem during single talker segments.
  arXiv  Detail & Related papers  (2021-03-03T13:01:08Z)
• FragmentVC: Any-to-Any Voice Conversion by End-to-End Extracting and Fusing Fine-Grained Voice Fragments With Attention [66.77490220410249]
  We propose FragmentVC, in which the latent phonetic structure of the utterance from the source speaker is obtained from Wav2Vec 2.0. FragmentVC is able to extract fine-grained voice fragments from the target speaker utterance(s) and fuse them into the desired utterance. This approach is trained with reconstruction loss only without any disentanglement considerations between content and speaker information.
  arXiv  Detail & Related papers  (2020-10-27T09:21:03Z)
• Semi-supervised Learning for Multi-speaker Text-to-speech Synthesis Using Discrete Speech Representation [125.59372403631006]
  We propose a semi-supervised learning approach for multi-speaker text-to-speech (TTS) A multi-speaker TTS model can learn from the untranscribed audio via the proposed encoder-decoder framework with discrete speech representation. We found the model can benefit from the proposed semi-supervised learning approach even when part of the unpaired speech data is noisy.
  arXiv  Detail & Related papers  (2020-05-16T15:47:11Z)
• Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario [51.50631198081903]
  We propose a novel Target-Speaker Voice Activity Detection (TS-VAD) approach. TS-VAD directly predicts an activity of each speaker on each time frame. Experiments on the CHiME-6 unsegmented data show that TS-VAD achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2020-05-14T21:24:56Z)
• SpEx: Multi-Scale Time Domain Speaker Extraction Network [89.00319878262005]
  Speaker extraction aims to mimic humans' selective auditory attention by extracting a target speaker's voice from a multi-talker environment. It is common to perform the extraction in frequency-domain, and reconstruct the time-domain signal from the extracted magnitude and estimated phase spectra. We propose a time-domain speaker extraction network (SpEx) that converts the mixture speech into multi-scale embedding coefficients instead of decomposing the speech signal into magnitude and phase spectra.
  arXiv  Detail & Related papers  (2020-04-17T16:13:06Z)
• Continuous speech separation: dataset and analysis [52.10378896407332]
  In natural conversations, a speech signal is continuous, containing both overlapped and overlap-free components. This paper describes a dataset and protocols for evaluating continuous speech separation algorithms.
  arXiv  Detail & Related papers  (2020-01-30T18:01:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.