X2Streaming-TTS: Causal Token-Level Text-to-Speech from Streaming Text with Speech-State Inheritance

• URL: http://arxiv.org/abs/2608.18661v1
• Date: Wed, 19 Aug 2026 08:05:45 GMT
• Title: X2Streaming-TTS: Causal Token-Level Text-to-Speech from Streaming Text with Speech-State Inheritance
• Abstract summary: Streaming text-to-speech is essential for low-latency spoken dialogue systems, but many systems wait for sentence-level text and are therefore only pseudo-streaming.<n>We present X2Streaming-TTS, a causal TTS framework that consumes asynchronously arriving text tokens and emits speech without accessing future input.
• Score: 4.713402367550747
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Streaming text-to-speech is essential for low-latency spoken dialogue systems, yet many systems wait for sentence-level text and are therefore only pseudo-streaming. True token-level synthesis must generate speech from uncertain prefixes while maintaining perceptual continuity over an unbounded stream with bounded context. We present X2Streaming-TTS, a causal TTS framework that consumes asynchronously arriving text tokens and emits speech without accessing future input. To handle uncertain prefixes, we introduce causal commitment, which keeps ambiguous expressions provisional through uncertainty-aware buffering and performs capacity-adaptive, punctuation-aware segmentation. To preserve acoustic continuity, we further introduce causal speech-state inheritance, which carries the complete Code2Wav state and selected historical Talker states across segment boundaries. Together with an attention prior constraint, it blocks access to future positions while retaining bounded acoustic context. Experiments show that X2Streaming-TTS outperforms existing pseudo-streaming models on most subjective and objective metrics. Further analysis shows that causal commitment stabilizes online segmentation and reduces failures caused by insufficient context, while speech-state inheritance improves boundary continuity without degrading naturalness or speaker identity. X2Streaming-TTS thus achieves strict token-level synthesis with quality comparable to the evaluated offline baselines, a median time to first audio token (TTFT) of 15.8 ms for a single request, and a median TTFT of 260.8 ms at 128 concurrent requests. Our implementation is publicly available at https://github.com/X-Square-Robot/X2Streaming-TTS .

Related papers

• MagpieTTS-LF: Inference-Time Long-Form Speech Generation Without Training on Long-Form data [13.076032296543032]
  We present an inference-time approach called MagpieTTS-LF that enables MagpieTTS to produce coherent long-form speech without model retraining.<n>Our method introduces three key innovations: (1) soft attention priors to guide monotonic alignment while preserving past and future context; (2) a stateful inference algorithm that maintains context across sentence chunks; and (3) history-aware text encoding that uses past text for discourse-level prosodic planning.
  arXiv  Detail & Related papers  (2026-06-16T20:58:26Z)
• Speech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation [48.98480497536377]
  We propose ELF-S2T, an audio-conditioned continuous-target generative model for speech-to-text systems.<n>ELF-S2T processes speech via a frozen Whisper encoder and a single linear projector, prepending the resulting audio condition to the noisy text latent.<n> Experiments on LibriSpeech and CoVoST2 show that ELF-S2T achieves competitive ASR and S2TT performance.
  arXiv  Detail & Related papers  (2026-06-09T03:27:30Z)
• Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input [37.127877690220934]
  Streaming TTS that receives streaming text is essential for interactive systems.<n>But this scheme faces two major challenges: unnatural prosody due to missing lookahead and long-form collapse due to unbounded context.<n>We propose a prosodic-boundary-aware post-training strategy, adapting a pretrained LLM-based TTS model using weakly time-aligned data.
  arXiv  Detail & Related papers  (2026-03-06T16:36:51Z)
• StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling [50.537794606598254]
  StreamMel is a pioneering single-stage streaming TTS framework that models continuous mel-spectrograms.<n>It enables low-latency, autoregressive synthesis while preserving high speaker similarity and naturalness.<n>It even achieves performance comparable to offline systems while supporting efficient real-time generation.
  arXiv  Detail & Related papers  (2025-06-14T16:53:39Z)
• SyncSpeech: Low-Latency and Efficient Dual-Stream Text-to-Speech based on Temporal Masked Transformer [68.78023656892319]
  This paper presents a dual-stream text-to-speech (TTS) model, SyncSpeech, capable of receiving streaming text input from upstream models while simultaneously generating streaming speech.<n>SyncSpeech has the following advantages: Low latency, as it begins generating streaming speech upon receiving the second text token; High efficiency, as it decodes all speech tokens corresponding to the each arrived text token in one step.
  arXiv  Detail & Related papers  (2025-02-16T12:14:17Z)
• A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Speech Translation [48.84039953531355]
  We propose a novel non-autoregressive generation framework for simultaneous speech translation (NAST-S2X) NAST-S2X integrates speech-to-text and speech-to-speech tasks into a unified end-to-end framework. It achieves high-quality simultaneous interpretation within a delay of less than 3 seconds and provides a 28 times decoding speedup in offline generation.
  arXiv  Detail & Related papers  (2024-06-11T04:25:48Z)
• StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection [23.75894159181602]
  Streaming speech-to-text translation (StreamST) is the task of automatically translating speech while incrementally receiving an audio stream. We introduce StreamAtt, the first StreamST policy, and propose StreamLAAL, the first StreamST latency metric.
  arXiv  Detail & Related papers  (2024-06-10T08:27:58Z)
• token2vec: A Joint Self-Supervised Pre-training Framework Using Unpaired Speech and Text [65.04385919645395]
  token2vec is a novel joint pre-training framework for unpaired speech and text based on discrete representations of speech. Experiments show that token2vec is significantly superior to various speech-only pre-training baselines, with up to 17.7% relative WER reduction.
  arXiv  Detail & Related papers  (2022-10-30T06:38:19Z)
• TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation [61.564874831498145]
  TranSpeech is a speech-to-speech translation model with bilateral perturbation. We establish a non-autoregressive S2ST technique, which repeatedly masks and predicts unit choices. TranSpeech shows a significant improvement in inference latency, enabling speedup up to 21.4x than autoregressive technique.
  arXiv  Detail & Related papers  (2022-05-25T06:34:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.