Leveraging speaker attribute information using multi task learning for speaker verification and diarization

• URL: http://arxiv.org/abs/2010.14269v2
• Date: Fri, 23 Apr 2021 16:45:16 GMT
• Title: Leveraging speaker attribute information using multi task learning for speaker verification and diarization
• Authors: Chau Luu, Peter Bell, Steve Renals
• Abstract summary: We propose a framework for making use of auxiliary label information, even when it is only available for speech corpora mismatched to the target application. We show that by leveraging two additional forms of speaker attribute information, we improve the performance of our deep speaker embeddings for both verification and diarization tasks.
• Score: 33.60058873783114
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep speaker embeddings have become the leading method for encoding speaker identity in speaker recognition tasks. The embedding space should ideally capture the variations between all possible speakers, encoding the multiple acoustic aspects that make up a speaker's identity, whilst being robust to non-speaker acoustic variation. Deep speaker embeddings are normally trained discriminatively, predicting speaker identity labels on the training data. We hypothesise that additionally predicting speaker-related auxiliary variables -- such as age and nationality -- may yield representations that are better able to generalise to unseen speakers. We propose a framework for making use of auxiliary label information, even when it is only available for speech corpora mismatched to the target application. On a test set of US Supreme Court recordings, we show that by leveraging two additional forms of speaker attribute information derived respectively from the matched training data, and VoxCeleb corpus, we improve the performance of our deep speaker embeddings for both verification and diarization tasks, achieving a relative improvement of 26.2% in DER and 6.7% in EER compared to baselines using speaker labels only. This improvement is obtained despite the auxiliary labels having been scraped from the web and being potentially noisy.

Related papers

• Identifying Speakers in Dialogue Transcripts: A Text-based Approach Using Pretrained Language Models [83.7506131809624]
  We introduce an approach to identifying speaker names in dialogue transcripts, a crucial task for enhancing content accessibility and searchability in digital media archives. We present a novel, large-scale dataset derived from the MediaSum corpus, encompassing transcripts from a wide range of media sources. We propose novel transformer-based models tailored for SpeakerID, leveraging contextual cues within dialogues to accurately attribute speaker names.
  arXiv  Detail & Related papers  (2024-07-16T18:03:58Z)
• Residual Information in Deep Speaker Embedding Architectures [4.619541348328938]
  This paper introduces an analysis over six sets of speaker embeddings extracted with some of the most recent and high-performing DNN architectures. The dataset includes 46 speakers uttering the same set of prompts, recorded in either a professional studio or their home environments. The results show that the discriminative power of the analyzed embeddings is very high, yet across all the analyzed architectures, residual information is still present in the representations.
  arXiv  Detail & Related papers  (2023-02-06T12:37:57Z)
• In search of strong embedding extractors for speaker diarisation [49.7017388682077]
  We tackle two key problems when adopting EEs for speaker diarisation. First, the evaluation is not straightforward because the features required for better performance differ between speaker verification and diarisation. We show that better performance on widely adopted speaker verification evaluation protocols does not lead to better diarisation performance. We propose two data augmentation techniques to alleviate the second problem, making embedding extractors aware of overlapped speech or speaker change input.
  arXiv  Detail & Related papers  (2022-10-26T13:00:29Z)
• Self-supervised Speaker Recognition Training Using Human-Machine Dialogues [22.262550043863445]
  We investigate how to pretrain speaker recognition models by leveraging dialogues between customers and smart-speaker devices. We propose an effective rejection mechanism that selectively learns from dialogues based on their acoustic homogeneity. Experiments demonstrate that the proposed method provides significant performance improvements, superior to earlier work.
  arXiv  Detail & Related papers  (2022-02-07T19:44:54Z)
• Improving on-device speaker verification using federated learning with privacy [5.321241042620525]
  Information on speaker characteristics can be useful as side information in improving speaker recognition accuracy. This paper investigates how privacy-preserving learning can improve a speaker verification system.
  arXiv  Detail & Related papers  (2020-08-06T13:37:14Z)
• Augmentation adversarial training for self-supervised speaker recognition [49.47756927090593]
  We train robust speaker recognition models without speaker labels. Experiments on VoxCeleb and VOiCES datasets show significant improvements over previous works using self-supervision.
  arXiv  Detail & Related papers  (2020-07-23T15:49:52Z)
• Joint Speaker Counting, Speech Recognition, and Speaker Identification for Overlapped Speech of Any Number of Speakers [38.3469744871394]
  We propose an end-to-end speaker-attributed automatic speech recognition model. It unifies speaker counting, speech recognition, and speaker identification on overlapped speech.
  arXiv  Detail & Related papers  (2020-06-19T02:05:18Z)
• Speaker Diarization with Lexical Information [59.983797884955]
  This work presents a novel approach for speaker diarization to leverage lexical information provided by automatic speech recognition. We propose a speaker diarization system that can incorporate word-level speaker turn probabilities with speaker embeddings into a speaker clustering process to improve the overall diarization accuracy.
  arXiv  Detail & Related papers  (2020-04-13T17:16:56Z)
• Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention [70.82604384963679]
  This paper investigates a self-adaptation method for speech enhancement using auxiliary speaker-aware features. We extract a speaker representation used for adaptation directly from the test utterance.
  arXiv  Detail & Related papers  (2020-02-14T05:05:36Z)
• Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam [100.95498268200777]
  SpeakerBeam exploits an adaptation utterance of the target speaker to extract his/her voice characteristics. SpeakerBeam sometimes fails when speakers have similar voice characteristics, such as in same-gender mixtures. We show experimentally that these strategies greatly improve speech extraction performance, especially for same-gender mixtures.
  arXiv  Detail & Related papers  (2020-01-23T05:36:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.