Multi-speaker Text-to-speech Training with Speaker Anonymized Data

• URL: http://arxiv.org/abs/2405.11767v1
• Date: Mon, 20 May 2024 03:55:44 GMT
• Title: Multi-speaker Text-to-speech Training with Speaker Anonymized Data
• Authors: Wen-Chin Huang, Yi-Chiao Wu, Tomoki Toda,
• Abstract summary: We investigate training multi-speaker text-to-speech (TTS) models using data that underwent speaker anonymization (SA) Two signal processing-based and three deep neural network-based SA methods were used to anonymize VCTK, a multi-speaker TTS dataset. We conducted extensive objective and subjective experiments to evaluate the anonymized training data, as well as the performance of the downstream TTS model trained using those data.
• Score: 40.70515431989197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The trend of scaling up speech generation models poses a threat of biometric information leakage of the identities of the voices in the training data, raising privacy and security concerns. In this paper, we investigate training multi-speaker text-to-speech (TTS) models using data that underwent speaker anonymization (SA), a process that tends to hide the speaker identity of the input speech while maintaining other attributes. Two signal processing-based and three deep neural network-based SA methods were used to anonymize VCTK, a multi-speaker TTS dataset, which is further used to train an end-to-end TTS model, VITS, to perform unseen speaker TTS during the testing phase. We conducted extensive objective and subjective experiments to evaluate the anonymized training data, as well as the performance of the downstream TTS model trained using those data. Importantly, we found that UTMOS, a data-driven subjective rating predictor model, and GVD, a metric that measures the gain of voice distinctiveness, are good indicators of the downstream TTS performance. We summarize insights in the hope of helping future researchers determine the goodness of the SA system for multi-speaker TTS training.

Related papers

• SpoofCeleb: Speech Deepfake Detection and SASV In The Wild [76.71096751337888]
  SpoofCeleb is a dataset designed for Speech Deepfake Detection (SDD) and Spoofing-robust Automatic Speaker Verification (SASV) We utilize source data from real-world conditions and spoofing attacks generated by Text-To-Speech (TTS) systems also trained on the same real-world data. SpoofCeleb comprises over 2.5 million utterances from 1,251 unique speakers, collected under natural, real-world conditions.
  arXiv  Detail & Related papers  (2024-09-18T23:17:02Z)
• Cross-Dialect Text-To-Speech in Pitch-Accent Language Incorporating Multi-Dialect Phoneme-Level BERT [29.167336994990542]
  Cross-dialect text-to-speech (CD-TTS) is a task to synthesize learned speakers' voices in non-native dialects. We present a novel TTS model comprising three sub-modules to perform competitively at this task.
  arXiv  Detail & Related papers  (2024-09-11T13:40:27Z)
• Application of ASV for Voice Identification after VC and Duration Predictor Improvement in TTS Models [0.0]
  This paper presents a system for automatic speaker verification. The primary objective of our model is the extraction of embeddings from the target speaker's audio. This information is used in our multivoice TTS pipeline, which is currently under development.
  arXiv  Detail & Related papers  (2024-06-27T15:08:51Z)
• An Initial Investigation of Language Adaptation for TTS Systems under Low-resource Scenarios [76.11409260727459]
  This paper explores the language adaptation capability of ZMM-TTS, a recent SSL-based multilingual TTS system. We demonstrate that the similarity in phonetics between the pre-training and target languages, as well as the language category, affects the target language's adaptation performance.
  arXiv  Detail & Related papers  (2024-06-13T08:16:52Z)
• Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation [76.13334392868208]
  Direct speech-to-speech translation (S2ST) models suffer from data scarcity issues. In this work, we explore self-supervised pre-training with unlabeled speech data and data augmentation to tackle this issue.
  arXiv  Detail & Related papers  (2022-04-06T17:59:22Z)
• Adapting TTS models For New Speakers using Transfer Learning [12.46931609726818]
  Training neural text-to-speech (TTS) models for a new speaker typically requires several hours of high quality speech data. We propose transfer-learning guidelines for adapting high quality single-speaker TTS models for a new speaker, using only a few minutes of speech data.
  arXiv  Detail & Related papers  (2021-10-12T07:51:25Z)
• A study on the efficacy of model pre-training in developing neural text-to-speech system [55.947807261757056]
  This study aims to understand better why and how model pre-training can positively contribute to TTS system performance. It is found that the TTS system could achieve comparable performance when the pre-training data is reduced to 1/8 of its original size.
  arXiv  Detail & Related papers  (2021-10-08T02:09:28Z)
• Semi-supervised Learning for Multi-speaker Text-to-speech Synthesis Using Discrete Speech Representation [125.59372403631006]
  We propose a semi-supervised learning approach for multi-speaker text-to-speech (TTS) A multi-speaker TTS model can learn from the untranscribed audio via the proposed encoder-decoder framework with discrete speech representation. We found the model can benefit from the proposed semi-supervised learning approach even when part of the unpaired speech data is noisy.
  arXiv  Detail & Related papers  (2020-05-16T15:47:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.