CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment

• URL: http://arxiv.org/abs/2602.19574v1
• Date: Mon, 23 Feb 2026 07:44:14 GMT
• Title: CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment
• Abstract summary: Large--model (LLM)-based text-to-speech (TTS) systems can generate natural speech, but most are not designed for low-latency dual-streaming synthesis.<n>We propose CTC-TTS, which replaces MFA with a CTC based aligner and introduces a bi-word based interleaving strategy.<n>Experiments show that CTC-TTS outperforms fixed-ratio interleaving and MFA-based baselines on streaming synthesis and zero-shot tasks.
• Score: 10.846147104273959
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-language-model (LLM)-based text-to-speech (TTS) systems can generate natural speech, but most are not designed for low-latency dual-streaming synthesis. High-quality dual-streaming TTS depends on accurate text--speech alignment and well-designed training sequences that balance synthesis quality and latency. Prior work often relies on GMM-HMM based forced-alignment toolkits (e.g., MFA), which are pipeline-heavy and less flexible than neural aligners; fixed-ratio interleaving of text and speech tokens struggles to capture text--speech alignment regularities. We propose CTC-TTS, which replaces MFA with a CTC based aligner and introduces a bi-word based interleaving strategy. Two variants are designed: CTC-TTS-L (token concatenation along the sequence length) for higher quality and CTC-TTS-F (embedding stacking along the feature dimension) for lower latency. Experiments show that CTC-TTS outperforms fixed-ratio interleaving and MFA-based baselines on streaming synthesis and zero-shot tasks. Speech samples are available at https://ctctts.github.io/.

Related papers

• DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech [15.682357754148548]
  Autoregressive (AR) text-to-speech (TTS) models generate discrete speech tokens sequentially, which makes inference slow and can degrade robustness by propagating local errors and hallucinations.<n>In this paper, we introduce DELTA-TTS, a lightweight LoRA-based adaptation framework that converts a pretrained AR TTS model into a discrete diffusion language model (dLLM) for confidence-ordered decoding.<n>To better capture the local structure of speech, DELTA-TTS incorporates a convolution module that injects local acoustic context, together with a $1/t$-weighted training objective and a time-shifted inference schedule
  arXiv  Detail & Related papers  (2026-07-05T06:45:47Z)
• TASTE-Streaming: Towards Streamable Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling [52.519745466772825]
  Text-speech joint spoken language modeling aims at natural and intelligent speech-based interactions.<n>TASTE produces speech tokens that align in lengths with their textual counterparts.<n>We propose TASTE-S, a streamable extension of TASTE suitable for real-time usage.
  arXiv  Detail & Related papers  (2026-03-12T18:13:48Z)
• Qwen3-TTS Technical Report [64.94647392030824]
  We present the Qwen3-TTS series, a family of advanced multilingual, controllable, robust, and streaming text-to-speech models.<n>Qwen3-TTS supports state-of-the-art 3-second voice cloning and description-based control.<n>Qwen3-TTS adopts a dual-track LM architecture for real-time synthesis, coupled with two speech tokenizers.
  arXiv  Detail & Related papers  (2026-01-22T03:51:43Z)
• Revisiting Direct Speech-to-Text Translation with Speech LLMs: Better Scaling than CoT Prompting? [13.202203902821333]
  We systematically compare Chain-of-Thought (CoT) and Direct prompting under increasing amounts of Speech-to-Text Translation (S2TT) data.<n>Our results show that Direct improves more consistently as the amount of data increases, suggesting that it may become a more effective approach as larger S2TT resources are created.
  arXiv  Detail & Related papers  (2025-10-03T15:23:32Z)
• GOAT-TTS: Expressive and Realistic Speech Generation via A Dual-Branch LLM [42.93855899824886]
  We propose a text-to-speech generation approach optimized via a novel dual-branch ArchiTecture (GOAT-TTS)<n>GOAT-TTS combines a speech encoder and projector to capture continuous acoustic embeddings, enabling bidirectional correlation between paralinguistic features (language, timbre, emotion) and semantic text representations without transcript dependency.<n> Experimental results demonstrate that our GOAT-TTS achieves performance comparable to state-of-the-art TTS models.
  arXiv  Detail & Related papers  (2025-04-15T01:44:56Z)
• SupertonicTTS: Towards Highly Efficient and Streamlined Text-to-Speech System [10.156665325783583]
  SupertonicTTS is a novel text-to-speech (TTS) system designed for efficient and streamlined speech synthesis.<n>We employ a low-dimensional latent space, temporal compression of latents, and ConvNeXt blocks to enable a lightweight architecture.<n> Experimental results demonstrate that SupertonicTTS delivers performance comparable to contemporary zero-shot TTS models with only 44M parameters.
  arXiv  Detail & Related papers  (2025-03-29T14:59:32Z)
• Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens [31.575335190916995]
  We introduce Spark-TTS, a novel system powered by BiCodec, a single-stream speech that decomposes speech into two complementary token types.<n>To facilitate research in controllable TTS, we introduce VoxBox, a meticulously curated 100,000-hour dataset with comprehensive attribute annotations.
  arXiv  Detail & Related papers  (2025-03-03T16:23:10Z)
• MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis [56.25862714128288]
  This paper introduces textitMegaTTS 3, a zero-shot text-to-speech (TTS) system featuring an innovative sparse alignment algorithm.<n>Specifically, we provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of alignment without limiting the search space.<n>Experiments demonstrate that MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports highly flexible control over accent intensity.
  arXiv  Detail & Related papers  (2025-02-26T08:22:00Z)
• Clip-TTS: Contrastive Text-content and Mel-spectrogram, A High-Quality Text-to-Speech Method based on Contextual Semantic Understanding [0.6798775532273751]
  I propose Clip-TTS, a TTS method based on the Clip architecture.<n>This method uses the Clip framework to establish a connection between text content and real mel-spectrograms during the text encoding stage.<n>In terms of model architecture, I adopt the basic structure of Transformer, which allows Clip-TTS to achieve fast inference speeds.
  arXiv  Detail & Related papers  (2025-02-26T07:09:33Z)
• SyncSpeech: Low-Latency and Efficient Dual-Stream Text-to-Speech based on Temporal Masked Transformer [68.78023656892319]
  This paper presents a dual-stream text-to-speech (TTS) model, SyncSpeech, capable of receiving streaming text input from upstream models while simultaneously generating streaming speech.<n>SyncSpeech has the following advantages: Low latency, as it begins generating streaming speech upon receiving the second text token; High efficiency, as it decodes all speech tokens corresponding to the each arrived text token in one step.
  arXiv  Detail & Related papers  (2025-02-16T12:14:17Z)
• Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation [76.13334392868208]
  Direct speech-to-speech translation (S2ST) models suffer from data scarcity issues. In this work, we explore self-supervised pre-training with unlabeled speech data and data augmentation to tackle this issue.
  arXiv  Detail & Related papers  (2022-04-06T17:59:22Z)
• AdaSpeech 2: Adaptive Text to Speech with Untranscribed Data [115.38309338462588]
  We develop AdaSpeech 2, an adaptive TTS system that only leverages untranscribed speech data for adaptation. Specifically, we introduce a mel-spectrogram encoder to a well-trained TTS model to conduct speech reconstruction. In adaptation, we use untranscribed speech data for speech reconstruction and only fine-tune the TTS decoder.
  arXiv  Detail & Related papers  (2021-04-20T01:53:30Z)
• MultiSpeech: Multi-Speaker Text to Speech with Transformer [145.56725956639232]
  Transformer-based text to speech (TTS) model (e.g., Transformer TTSciteli 2019neural, FastSpeechciteren 2019fastspeech) has shown the advantages of training and inference efficiency over RNN-based model. We develop a robust and high-quality multi-speaker Transformer TTS system called MultiSpeech, with several specially designed components/techniques to improve text-to-speech alignment.
  arXiv  Detail & Related papers  (2020-06-08T15:05:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.