Speaker Generation

• URL: http://arxiv.org/abs/2111.05095v1
• Date: Sun, 7 Nov 2021 22:31:41 GMT
• Title: Speaker Generation
• Authors: Daisy Stanton, Matt Shannon, Soroosh Mariooryad, RJ Skerry-Ryan, Eric Battenberg, Tom Bagby, David Kao
• Abstract summary: This work explores the task of synthesizing speech in nonexistent human-sounding voices. We present TacoSpawn, a system that performs competitively at this task.
• Score: 16.035697779803627
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work explores the task of synthesizing speech in nonexistent human-sounding voices. We call this task "speaker generation", and present TacoSpawn, a system that performs competitively at this task. TacoSpawn is a recurrent attention-based text-to-speech model that learns a distribution over a speaker embedding space, which enables sampling of novel and diverse speakers. Our method is easy to implement, and does not require transfer learning from speaker ID systems. We present objective and subjective metrics for evaluating performance on this task, and demonstrate that our proposed objective metrics correlate with human perception of speaker similarity. Audio samples are available on our demo page.

Related papers

• Identifying Speakers in Dialogue Transcripts: A Text-based Approach Using Pretrained Language Models [83.7506131809624]
  We introduce an approach to identifying speaker names in dialogue transcripts, a crucial task for enhancing content accessibility and searchability in digital media archives. We present a novel, large-scale dataset derived from the MediaSum corpus, encompassing transcripts from a wide range of media sources. We propose novel transformer-based models tailored for SpeakerID, leveraging contextual cues within dialogues to accurately attribute speaker names.
  arXiv  Detail & Related papers  (2024-07-16T18:03:58Z)
• PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptions [21.15647416266187]
  We propose PromptTTS++, a prompt-based text-to-speech (TTS) synthesis system that allows control over speaker identity using natural language descriptions. We introduce the concept of speaker prompt, which describes voice characteristics designed to be approximately independent of speaking style. Our subjective evaluation results show that the proposed method can better control speaker characteristics than the methods without the speaker prompt.
  arXiv  Detail & Related papers  (2023-09-15T04:11:37Z)
• ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech [58.93395189153713]
  We extend the pretraining method for cross-lingual multi-speaker speech synthesis tasks. We propose a speech-text joint pretraining framework, where we randomly mask the spectrogram and the phonemes. Our model shows great improvements over speaker-embedding-based multi-speaker TTS methods.
  arXiv  Detail & Related papers  (2022-11-07T13:35:16Z)
• Automatic Evaluation of Speaker Similarity [0.0]
  We introduce a new automatic evaluation method for speaker similarity assessment, consistent with human perceptual scores. Our experiments show that we can train a model to predict speaker similarity MUSHRA scores from speaker embeddings with 0.96 accuracy and significant correlation up to 0.78 Pearson score at the utterance level.
  arXiv  Detail & Related papers  (2022-07-01T11:23:16Z)
• Self supervised learning for robust voice cloning [3.7989740031754806]
  We use features learned in a self-supervised framework to produce high quality speech representations. The learned features are used as pre-trained utterance-level embeddings and as inputs to a Non-Attentive Tacotron based architecture. This method enables us to train our model in an unlabeled multispeaker dataset as well as use unseen speaker embeddings to copy a speaker's voice.
  arXiv  Detail & Related papers  (2022-04-07T13:05:24Z)
• Improved Relation Networks for End-to-End Speaker Verification and Identification [0.0]
  Speaker identification systems are tasked to identify a speaker amongst a set of enrolled speakers given just a few samples. We propose improved relation networks for speaker verification and few-shot (unseen) speaker identification. Inspired by the use of prototypical networks in speaker verification, we train the model to classify samples in the current episode amongst all speakers present in the training set.
  arXiv  Detail & Related papers  (2022-03-31T17:44:04Z)
• Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings [53.11450530896623]
  This paper presents a streaming speaker-attributed automatic speech recognition (SA-ASR) model that can recognize "who spoke what" Our model is based on token-level serialized output training (t-SOT) which was recently proposed to transcribe multi-talker speech in a streaming fashion. The proposed model achieves substantially better accuracy than a prior streaming model and shows comparable or sometimes even superior results to the state-of-the-art offline SA-ASR model.
  arXiv  Detail & Related papers  (2022-03-30T21:42:00Z)
• Speaker-Conditioned Hierarchical Modeling for Automated Speech Scoring [60.55025339250815]
  We propose a novel deep learning technique for non-native ASS, called speaker-conditioned hierarchical modeling. We take advantage of the fact that oral proficiency tests rate multiple responses for a candidate. In our technique, we take advantage of the fact that oral proficiency tests rate multiple responses for a candidate. We extract context from these responses and feed them as additional speaker-specific context to our network to score a particular response.
  arXiv  Detail & Related papers  (2021-08-30T07:00:28Z)
• Semi-supervised Learning for Multi-speaker Text-to-speech Synthesis Using Discrete Speech Representation [125.59372403631006]
  We propose a semi-supervised learning approach for multi-speaker text-to-speech (TTS) A multi-speaker TTS model can learn from the untranscribed audio via the proposed encoder-decoder framework with discrete speech representation. We found the model can benefit from the proposed semi-supervised learning approach even when part of the unpaired speech data is noisy.
  arXiv  Detail & Related papers  (2020-05-16T15:47:11Z)
• From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraint [11.982748481062542]
  This paper presents a system involving feedback constraint for multispeaker speech synthesis. We manage to enhance the knowledge transfer from the speaker verification to the speech synthesis by engaging the speaker verification network. The model is trained and evaluated on publicly available datasets.
  arXiv  Detail & Related papers  (2020-05-10T06:11:37Z)
• Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam [100.95498268200777]
  SpeakerBeam exploits an adaptation utterance of the target speaker to extract his/her voice characteristics. SpeakerBeam sometimes fails when speakers have similar voice characteristics, such as in same-gender mixtures. We show experimentally that these strategies greatly improve speech extraction performance, especially for same-gender mixtures.
  arXiv  Detail & Related papers  (2020-01-23T05:36:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.