Bagpiper-TTS: Natural Language Guided Universal Speech Synthesis

• URL: http://arxiv.org/abs/2606.22811v1
• Date: Mon, 22 Jun 2026 03:40:28 GMT
• Title: Bagpiper-TTS: Natural Language Guided Universal Speech Synthesis
• Abstract summary: Bagpiper-TTS is a universal speech synthesis system that deals with diverse natural language user requests.<n>Our model inherently supports a broad spectrum of tasks besides classical TTS applications.
• Score: 73.41484972991475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Classical TTS systems typically rely on rigid input formats and predefined metadata slots, limiting their ability to fulfill flexible user requirements. This paper introduces Bagpiper-TTS, a universal speech synthesis system that deals with diverse natural language user requests. Given a natural language prompt, Bagpiper-TTS first reasons over the users' intent to derive a rich caption, i.e., a comprehensive textual blueprint encompassing both transcription and nuanced metadata. Subsequently, this caption guides the synthesis of the target speech. Our model inherently supports a broad spectrum of tasks besides classical TTS applications, including multi-talker, intent-to-speech, role-play synthesis, singing voice synthesis, and more. Experimental results demonstrate that Bagpiper-TTS achieves an 1.7% Word Error Rate (WER) on the Seed-TTS-Eval benchmark and match the performance of dedicated models in both LLM-as-a-judge and human subjective evaluations across multiple applications.

Related papers

• Borderless Long Speech Synthesis [30.36601404142387]
  We introduce the Borderless Long Speech Synthesis framework for agent-centric, borderless long audio synthesis.<n>Rather than targeting a single narrow task, the system is designed as a unified capability set spanning VoiceDesigner, multi-speaker synthesis, Instruct TTS, and long-form text synthesis.
  arXiv  Detail & Related papers  (2026-03-20T09:37:54Z)
• CapSpeech: Enabling Downstream Applications in Style-Captioned Text-to-Speech [66.29338461991715]
  We introduce CapSpeech, a new benchmark for a series of CapTTS-related tasks.<n>CapSpeech comprises over 10 million machine-annotated audio-caption pairs and nearly 0.36 million human-annotated audio-caption pairs.<n>We conduct comprehensive experiments using both autoregressive and non-autoregressive models on CapSpeech.
  arXiv  Detail & Related papers  (2025-06-03T13:28:55Z)
• GOAT-TTS: Expressive and Realistic Speech Generation via A Dual-Branch LLM [42.93855899824886]
  We propose a text-to-speech generation approach optimized via a novel dual-branch ArchiTecture (GOAT-TTS)<n>GOAT-TTS combines a speech encoder and projector to capture continuous acoustic embeddings, enabling bidirectional correlation between paralinguistic features (language, timbre, emotion) and semantic text representations without transcript dependency.<n> Experimental results demonstrate that our GOAT-TTS achieves performance comparable to state-of-the-art TTS models.
  arXiv  Detail & Related papers  (2025-04-15T01:44:56Z)
• Text-To-Speech Synthesis In The Wild [76.71096751337888]
  We introduce the TTS In the Wild dataset, created through a fully automated pipeline applied to the VoxCeleb1 dataset.<n>It comprises two training sets: TITW-Hard, derived from the transcription, segmentation, and selection of raw VoxCeleb1 data, and TITW-Easy, which incorporates additional enhancement and data selection based on DNSMOS.
  arXiv  Detail & Related papers  (2024-09-13T10:58:55Z)
• LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioning [12.069474749489897]
  We introduce LibriTTS-P, a new corpus based on LibriTTS-R that includes utterance-level descriptions (i.e., prompts) of speaking style and speaker-level prompts of speaker characteristics. Results for style captioning tasks show that the model utilizing LibriTTS-P generates 2.5 times more accurate words than the model using a conventional dataset.
  arXiv  Detail & Related papers  (2024-06-12T07:49:21Z)
• Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias [71.94109664001952]
  Mega-TTS is a novel zero-shot TTS system that is trained with large-scale wild data. We show that Mega-TTS surpasses state-of-the-art TTS systems on zero-shot TTS speech editing, and cross-lingual TTS tasks.
  arXiv  Detail & Related papers  (2023-06-06T08:54:49Z)
• M2-CTTS: End-to-End Multi-scale Multi-modal Conversational Text-to-Speech Synthesis [38.85861825252267]
  M2-CTTS aims to comprehensively utilize historical conversation and enhance prosodic expression. We design a textual context module and an acoustic context module with both coarse-grained and fine-grained modeling.
  arXiv  Detail & Related papers  (2023-05-03T16:59:38Z)
• Enhancing Speech-to-Speech Translation with Multiple TTS Targets [62.18395387305803]
  We analyze the effect of changing synthesized target speech for direct S2ST models. We propose a multi-task framework that jointly optimized the S2ST system with multiple targets from different TTS systems.
  arXiv  Detail & Related papers  (2023-04-10T14:33:33Z)
• A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech [94.64927912924087]
  We train TTS systems using real-world speech from YouTube and podcasts. Recent Text-to-Speech architecture is designed for multiple code generation and monotonic alignment. We show thatRecent Text-to-Speech architecture outperforms existing TTS systems in several objective and subjective measures.
  arXiv  Detail & Related papers  (2023-02-08T17:34:32Z)
• Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech [88.22544315633687]
  Polyphone disambiguation aims to capture accurate pronunciation knowledge from natural text sequences for reliable Text-to-speech systems. We propose Dict-TTS, a semantic-aware generative text-to-speech model with an online website dictionary. Experimental results in three languages show that our model outperforms several strong baseline models in terms of pronunciation accuracy.
  arXiv  Detail & Related papers  (2022-06-05T10:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.