Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input

• URL: http://arxiv.org/abs/2603.06444v1
• Date: Fri, 06 Mar 2026 16:36:51 GMT
• Title: Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input
• Abstract summary: Streaming TTS that receives streaming text is essential for interactive systems.<n>But this scheme faces two major challenges: unnatural prosody due to missing lookahead and long-form collapse due to unbounded context.<n>We propose a prosodic-boundary-aware post-training strategy, adapting a pretrained LLM-based TTS model using weakly time-aligned data.
• Score: 37.127877690220934
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Streaming TTS that receives streaming text is essential for interactive systems, yet this scheme faces two major challenges: unnatural prosody due to missing lookahead and long-form collapse due to unbounded context. We propose a prosodic-boundary-aware post-training strategy, adapting a pretrained LLM-based TTS model using weakly time-aligned data. Specifically, the model is adapted to learn early stopping at specified content boundaries when provided with limited future text. During inference, a sliding-window prompt carries forward previous text and speech tokens, ensuring bounded context and seamless concatenation. Evaluations show our method outperforms CosyVoice-Style interleaved baseline in both short and long-form scenarios. In long-text synthesis, especially, it achieves a 66.2% absolute reduction in word error rate (from 71.0% to 4.8%) and increases speaker and emotion similarity by 16.1% and 1.5% relatively, offering a robust solution for streaming TTS with incremental text.

Related papers

• Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation [16.8983983115275]
  Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet poorly suited for spoken delivery via Text-to-Speech (TTS)<n>In this work, we study how to make LLMs generate TTS-friendly text, which we frame as a preference alignment problem.<n>We introduce two preference datasets spanning different target domains, CORA and Recipe, which contain paired TTS-friendly and TTS-unfriendly responses.
  arXiv  Detail & Related papers  (2026-09-01T13:44:43Z)
• X2Streaming-TTS: Causal Token-Level Text-to-Speech from Streaming Text with Speech-State Inheritance [4.713402367550747]
  Streaming text-to-speech is essential for low-latency spoken dialogue systems, but many systems wait for sentence-level text and are therefore only pseudo-streaming.<n>We present X2Streaming-TTS, a causal TTS framework that consumes asynchronously arriving text tokens and emits speech without accessing future input.
  arXiv  Detail & Related papers  (2026-08-19T08:05:45Z)
• VoXtream2: Full-stream TTS with dynamic speaking rate control [17.067283475630095]
  VoXtream2 is a zero-shot full-stream text-to-speech model with dynamic speaking-rate control.<n>Prompt-text masking enables textless audio prompting, removing the need for prompt transcription.<n>In full-stream mode, it runs 4 times faster than real time with 74 ms first-packet latency on a consumer GPU.
  arXiv  Detail & Related papers  (2026-03-13T18:49:25Z)
• Adaptive Duration Model for Text Speech Alignment [2.594813802197567]
  Speech-to-text alignment is a critical component of neural text to speech (TTS) models.<n>We propose a novel duration prediction framework that can give promising phoneme-level duration distribution with given text.
  arXiv  Detail & Related papers  (2025-07-30T12:31:11Z)
• StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling [50.537794606598254]
  StreamMel is a pioneering single-stage streaming TTS framework that models continuous mel-spectrograms.<n>It enables low-latency, autoregressive synthesis while preserving high speaker similarity and naturalness.<n>It even achieves performance comparable to offline systems while supporting efficient real-time generation.
  arXiv  Detail & Related papers  (2025-06-14T16:53:39Z)
• Zero-Shot Streaming Text to Speech Synthesis with Transducer and Auto-Regressive Modeling [76.23539797803681]
  Existing methods primarily use a look mechanism, relying on future text to achieve natural streaming speech synthesis.<n>We propose LE, a streaming framework for generating high-quality speech frame-by-frame.<n> Experimental results suggest that the LE outperforms current streaming TTS methods and achieves comparable performance over sentence-level TTS systems.
  arXiv  Detail & Related papers  (2025-05-26T08:25:01Z)
• SupertonicTTS: Towards Highly Efficient and Streamlined Text-to-Speech System [10.156665325783583]
  SupertonicTTS is a novel text-to-speech (TTS) system designed for efficient and streamlined speech synthesis.<n>We employ a low-dimensional latent space, temporal compression of latents, and ConvNeXt blocks to enable a lightweight architecture.<n> Experimental results demonstrate that SupertonicTTS delivers performance comparable to contemporary zero-shot TTS models with only 44M parameters.
  arXiv  Detail & Related papers  (2025-03-29T14:59:32Z)
• MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis [56.25862714128288]
  This paper introduces textitMegaTTS 3, a zero-shot text-to-speech (TTS) system featuring an innovative sparse alignment algorithm.<n>Specifically, we provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of alignment without limiting the search space.<n>Experiments demonstrate that MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports highly flexible control over accent intensity.
  arXiv  Detail & Related papers  (2025-02-26T08:22:00Z)
• DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors [8.419383213705789]
  We introduce DiTTo-TTS, a Diffusion Transformer (DiT)-based TTS model, to investigate whether LDM-based TTS can achieve state-of-the-art performance without domain-specific factors.<n>We find that DiT with minimal modifications outperforms U-Net, variable-length modeling with a speech length predictor, and conditions like semantic alignment in speech latent representations are key to further enhancement.
  arXiv  Detail & Related papers  (2024-06-17T11:25:57Z)
• Code-Switching Text Generation and Injection in Mandarin-English ASR [57.57570417273262]
  We investigate text generation and injection for improving the performance of an industry commonly-used streaming model, Transformer-Transducer (T-T) We first propose a strategy to generate code-switching text data and then investigate injecting generated text into T-T model explicitly by Text-To-Speech (TTS) conversion or implicitly by tying speech and text latent spaces. Experimental results on the T-T model trained with a dataset containing 1,800 hours of real Mandarin-English code-switched speech show that our approaches to inject generated code-switching text significantly boost the performance of T-T models.
  arXiv  Detail & Related papers  (2023-03-20T09:13:27Z)
• Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narration [62.75234183218897]
  We propose a one-stage context-aware framework to generate natural and coherent target speech without any training data of the speaker. We generate the mel-spectrogram of the edited speech with a transformer-based decoder. It outperforms a recent zero-shot TTS engine by a large margin.
  arXiv  Detail & Related papers  (2021-09-12T04:17:53Z)
• End-to-End Text-to-Speech using Latent Duration based on VQ-VAE [48.151894340550385]
  Explicit duration modeling is a key to achieving robust and efficient alignment in text-to-speech synthesis (TTS) We propose a new TTS framework using explicit duration modeling that incorporates duration as a discrete latent variable to TTS.
  arXiv  Detail & Related papers  (2020-10-19T15:34:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.