VoXtream2: Full-stream TTS with dynamic speaking rate control

• URL: http://arxiv.org/abs/2603.13518v1
• Date: Fri, 13 Mar 2026 18:49:25 GMT
• Title: VoXtream2: Full-stream TTS with dynamic speaking rate control
• Abstract summary: VoXtream2 is a zero-shot full-stream text-to-speech model with dynamic speaking-rate control.<n>Prompt-text masking enables textless audio prompting, removing the need for prompt transcription.<n>In full-stream mode, it runs 4 times faster than real time with 74 ms first-packet latency on a consumer GPU.
• Score: 17.067283475630095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Full-stream text-to-speech (TTS) for interactive systems must start speaking with minimal delay while remaining controllable as text arrives incrementally. We present VoXtream2, a zero-shot full-stream TTS model with dynamic speaking-rate control that can be updated mid-utterance on the fly. VoXtream2 combines a distribution matching mechanism over duration states with classifier-free guidance across conditioning signals to improve controllability and synthesis quality. Prompt-text masking enables textless audio prompting, removing the need for prompt transcription. Across standard zero-shot benchmarks and a dedicated speaking-rate test set, VoXtream2 achieves competitive objective and subjective results against public baselines despite a smaller model and less training data. In full-stream mode, it runs 4 times faster than real time with 74 ms first-packet latency on a consumer GPU.

Related papers

• VoiceChat-TTS: A Low-Latency Continuous Speech Synthesis Model for Interactive Agents [31.302856029488876]
  VoiceChat-TTS is a low-latency, continuous, and streamable text-to-speech model for interactive agents.<n>The model enables always-on, responsive speech generation while preserving modularity and high speech quality.
  arXiv  Detail & Related papers  (2026-08-13T23:37:07Z)
• CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents [53.73961877379327]
  CuteTTS is a compact continuous-autoregressive TTS system.<n>It combines semantically aligned causal VAE latents with patch-level autoregression, explicit speaker conditioning, and a bidirectional flow-matching head.
  arXiv  Detail & Related papers  (2026-08-09T11:10:20Z)
• Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Input [37.127877690220934]
  Streaming TTS that receives streaming text is essential for interactive systems.<n>But this scheme faces two major challenges: unnatural prosody due to missing lookahead and long-form collapse due to unbounded context.<n>We propose a prosodic-boundary-aware post-training strategy, adapting a pretrained LLM-based TTS model using weakly time-aligned data.
  arXiv  Detail & Related papers  (2026-03-06T16:36:51Z)
• Voxtral Realtime [134.66962524291424]
  Voxtral Realtime is a streaming automatic speech recognition model.<n>It matches offline transcription quality at sub-second latency.<n>We release the model weights under the Apache 2.0 license.
  arXiv  Detail & Related papers  (2026-02-11T19:17:10Z)
• VoXtream: Full-Stream Text-to-Speech with Extremely Low Latency [17.067283475630095]
  We present VoXtream, a fully autoregressive, zero-shot streaming text-to-speech (TTS) system for real-time use that begins speaking from the first word.<n>VoXtream directly maps incoming phonemes to audio tokens using a monotonic alignment scheme and a dynamic look-ahead that does not delay onset.<n>Built around an incremental phoneme transformer, a temporal transformer predicting semantic and duration tokens, and a depth transformer producing acoustic tokens, VoXtream achieves, to our knowledge, the lowest initial delay among publicly available streaming TTS: 102 ms on
  arXiv  Detail & Related papers  (2025-09-19T13:26:46Z)
• StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling [50.537794606598254]
  StreamMel is a pioneering single-stage streaming TTS framework that models continuous mel-spectrograms.<n>It enables low-latency, autoregressive synthesis while preserving high speaker similarity and naturalness.<n>It even achieves performance comparable to offline systems while supporting efficient real-time generation.
  arXiv  Detail & Related papers  (2025-06-14T16:53:39Z)
• Zero-Shot Streaming Text to Speech Synthesis with Transducer and Auto-Regressive Modeling [76.23539797803681]
  Existing methods primarily use a look mechanism, relying on future text to achieve natural streaming speech synthesis.<n>We propose LE, a streaming framework for generating high-quality speech frame-by-frame.<n> Experimental results suggest that the LE outperforms current streaming TTS methods and achieves comparable performance over sentence-level TTS systems.
  arXiv  Detail & Related papers  (2025-05-26T08:25:01Z)
• SpeakStream: Streaming Text-to-Speech with Interleaved Data [11.131427505801062]
  We present SpeakStream, a streaming TTS system that generates audio incrementally from streaming text using a decoder-only architecture.<n>During inference, SpeakStream generates speech incrementally while absorbing streaming input text.<n>Our experiments demonstrate that SpeakStream achieves state-of-the-art latency while maintaining the quality of non-streaming TTS systems.
  arXiv  Detail & Related papers  (2025-05-25T16:11:10Z)
• SyncSpeech: Low-Latency and Efficient Dual-Stream Text-to-Speech based on Temporal Masked Transformer [68.78023656892319]
  This paper presents a dual-stream text-to-speech (TTS) model, SyncSpeech, capable of receiving streaming text input from upstream models while simultaneously generating streaming speech.<n>SyncSpeech has the following advantages: Low latency, as it begins generating streaming speech upon receiving the second text token; High efficiency, as it decodes all speech tokens corresponding to the each arrived text token in one step.
  arXiv  Detail & Related papers  (2025-02-16T12:14:17Z)
• ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control [50.27383290553548]
  ControlSpeech is a text-to-speech (TTS) system capable of fully cloning the speaker's voice and enabling arbitrary control and adjustment of speaking style.<n>We show that ControlSpeech exhibits comparable or state-of-the-art (SOTA) performance in terms of controllability, timbre similarity, audio quality, robustness, and generalizability.
  arXiv  Detail & Related papers  (2024-06-03T11:15:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.