LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning

• URL: http://arxiv.org/abs/2406.07969v1
• Date: Wed, 12 Jun 2024 07:49:21 GMT
• Title: LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning
• Authors: Masaya Kawamura, Ryuichi Yamamoto, Yuma Shirahata, Takuya Hasumi, Kentaro Tachibana,
• Abstract summary: We introduce LibriTTS-P, a new corpus based on LibriTTS-R that includes utterance-level descriptions (i.e., prompts) of speaking style and speaker-level prompts of speaker characteristics. Results for style captioning tasks show that the model utilizing LibriTTS-P generates 2.5 times more accurate words than the model using a conventional dataset.
• Score: 12.069474749489897
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce LibriTTS-P, a new corpus based on LibriTTS-R that includes utterance-level descriptions (i.e., prompts) of speaking style and speaker-level prompts of speaker characteristics. We employ a hybrid approach to construct prompt annotations: (1) manual annotations that capture human perceptions of speaker characteristics and (2) synthetic annotations on speaking style. Compared to existing English prompt datasets, our corpus provides more diverse prompt annotations for all speakers of LibriTTS-R. Experimental results for prompt-based controllable TTS demonstrate that the TTS model trained with LibriTTS-P achieves higher naturalness than the model using the conventional dataset. Furthermore, the results for style captioning tasks show that the model utilizing LibriTTS-P generates 2.5 times more accurate words than the model using a conventional dataset. Our corpus, LibriTTS-P, is available at https://github.com/line/LibriTTS-P.

Related papers

• Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data? [49.42189569058647]
  Two-pass direct speech-to-speech translation (S2ST) models decompose the task into speech-to-text translation (S2TT) and text-to-speech (TTS) In this paper, we introduce a composite S2ST model named ComSpeech, which can seamlessly integrate any pretrained S2TT and TTS models into a direct S2ST model. We also propose a novel training method ComSpeech-ZS that solely utilizes S2TT and TTS data.
  arXiv  Detail & Related papers  (2024-06-11T14:17:12Z)
• Expressive TTS Driven by Natural Language Prompts Using Few Human Annotations [12.891344121936902]
  Expressive text-to-speech (TTS) aims to synthesize speeches with human-like tones, moods, or even artistic attributes. Recent advancements in TTS empower users with the ability to directly control synthesis style through natural language prompts. We present FreeStyleTTS (FS-TTS), a controllable expressive TTS model with minimal human annotations.
  arXiv  Detail & Related papers  (2023-11-02T14:20:37Z)
• TextrolSpeech: A Text Style Control Speech Corpus With Codec Language Text-to-Speech Models [51.529485094900934]
  We propose TextrolSpeech, which is the first large-scale speech emotion dataset annotated with rich text attributes. We introduce a multi-stage prompt programming approach that effectively utilizes the GPT model for generating natural style descriptions in large volumes. To address the need for generating audio with greater style diversity, we propose an efficient architecture called Salle.
  arXiv  Detail & Related papers  (2023-08-28T09:06:32Z)
• DSE-TTS: Dual Speaker Embedding for Cross-Lingual Text-to-Speech [30.110058338155675]
  Cross-lingual text-to-speech (CTTS) is still far from satisfactory as it is difficult to accurately retain the speaker timbres. We propose a novel dual speaker embedding TTS (DSE-TTS) framework for CTTS with authentic speaking style. By combining both embeddings, DSE-TTS significantly outperforms the state-of-the-art SANE-TTS in cross-lingual synthesis.
  arXiv  Detail & Related papers  (2023-06-25T06:46:36Z)
• Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias [71.94109664001952]
  Mega-TTS is a novel zero-shot TTS system that is trained with large-scale wild data. We show that Mega-TTS surpasses state-of-the-art TTS systems on zero-shot TTS speech editing, and cross-lingual TTS tasks.
  arXiv  Detail & Related papers  (2023-06-06T08:54:49Z)
• ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpus [3.1925030748447747]
  We present a speech corpus for Classical Arabic Text-to-Speech (ClArTTS) to support the development of end-to-end TTS systems for Arabic. The speech is extracted from a LibriVox audiobook, which is then processed, segmented, and manually transcribed and annotated. The final ClArTTS corpus contains about 12 hours of speech from a single male speaker sampled at 40100 kHz.
  arXiv  Detail & Related papers  (2023-02-28T20:18:59Z)
• Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech [88.22544315633687]
  Polyphone disambiguation aims to capture accurate pronunciation knowledge from natural text sequences for reliable Text-to-speech systems. We propose Dict-TTS, a semantic-aware generative text-to-speech model with an online website dictionary. Experimental results in three languages show that our model outperforms several strong baseline models in terms of pronunciation accuracy.
  arXiv  Detail & Related papers  (2022-06-05T10:50:34Z)
• Transfer Learning Framework for Low-Resource Text-to-Speech using a Large-Scale Unlabeled Speech Corpus [10.158584616360669]
  Training a text-to-speech (TTS) model requires a large scale text labeled speech corpus. We propose a transfer learning framework for TTS that utilizes a large amount of unlabeled speech dataset for pre-training.
  arXiv  Detail & Related papers  (2022-03-29T11:26:56Z)
• Textless Speech-to-Speech Translation on Real Data [49.134208897722246]
  We present a textless speech-to-speech translation (S2ST) system that can translate speech from one language into another language. We tackle the challenge in modeling multi-speaker target speech and train the systems with real-world S2ST data.
  arXiv  Detail & Related papers  (2021-12-15T18:56:35Z)
• Referee: Towards reference-free cross-speaker style transfer with low-quality data for expressive speech synthesis [39.730034713382736]
  Cross-speaker style transfer (CSST) in text-to-speech (TTS) aims at transferring a speaking style to the synthesised speech in a target speaker's voice. This presents Referee, a robust reference-free CSST approach for expressive TTS, which fully leverages low-quality data to learn speaking styles from text.
  arXiv  Detail & Related papers  (2021-09-08T05:39:34Z)
• Learning Speaker Embedding from Text-to-Speech [59.80309164404974]
  We jointly trained end-to-end Tacotron 2 TTS and speaker embedding networks in a self-supervised fashion. We investigated training TTS from either manual or ASR-generated transcripts. Unsupervised TTS embeddings improved EER by 2.06% absolute with regard to i-vectors for the LibriTTS dataset.
  arXiv  Detail & Related papers  (2020-10-21T18:03:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.