Faster IndexTTS-2: Accelerating and Streaming Autoregressive Zero-Shot Text-to-Speech Synthesis on GPUs

• URL: http://arxiv.org/abs/2607.21042v1
• Date: Thu, 23 Jul 2026 08:24:21 GMT
• Title: Faster IndexTTS-2: Accelerating and Streaming Autoregressive Zero-Shot Text-to-Speech Synthesis on GPUs
• Abstract summary: IndexTTS-2 is a state-of-the-art autoregressive TTS model consisting of a GPT, a flow-matching Diffusion Transformer, and a vocoder.<n>Despite its high quality, its inference speed barely reaches real-time without streaming or support.<n>We present Faster IndexTTS-2, which accelerates all neural components of IndexTTS-2 for production deployment on GPUs.
• Score: 6.875322067946555
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autoregressive text-to-speech models achieve strong naturalness but suffer from slow inference due to sequential token generation, limiting their deployment in production applications that require low latency. IndexTTS-2 is a state-of-the-art autoregressive TTS model consisting of a GPT, a flow-matching Diffusion Transformer, and a vocoder. Despite its high synthesis quality, its inference speed barely reaches real-time without streaming or batching support. We present Faster IndexTTS-2, which accelerates all neural network components of IndexTTS-2 for production deployment on GPUs using NVIDIA TensorRT and TensorRT-LLM. Faster IndexTTS-2 also enables streaming synthesis for latency-sensitive interactive applications, and batched inference across all components to maximize GPU utilization. Experiments on the Seed-TTS benchmark for both English and Chinese demonstrate up to 5.0$\times$ speedup on the autoregressive GPT and 3.6$\times$ end-to-end, with minimal degradation in word error rate, speaker similarity, and naturalness. Our methodology provides a practical reference for efficiently accelerating similar autoregressive speech models on GPUs.

Related papers

• TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation [105.14745366152725]
  TurboT2VA is a distillation and inference framework for joint text-to-video-audio generation.<n>On LTX-2, four-step distillation reduces generator latency from 50.52s to 2.51s at the standard evaluation resolution of 512$times$768.<n>The complete stack reduces generator latency from 318.74s to 5.83s on one NVIDIA H20, achieving a 54.67$times$ generator-only speedup.
  arXiv  Detail & Related papers  (2026-08-25T15:09:02Z)
• FreyaTTS Technical Report [0.0]
  We introduce Freya-TTS, a compact, tokenizer-free, Turkish-first text-to-speaker model.<n>It operates in the frozen continuous latent space of AudioVAE2 (16 kHz encode, 48 kHz decode), allowing the model to focus its capacity on text-to-latent mapping.<n>It achieves a band-matched word error rate (WER) of 8.0% and character error rate (CER) of 3.0%, outperforming substantially larger open-source systems.
  arXiv  Detail & Related papers  (2026-07-10T15:36:30Z)
• Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference [5.685908474026849]
  High-quality automatic speech recognition (ASR) on edge devices requires models that jointly optimize accuracy, latency, and memory footprint while operating entirely on CPU without GPU acceleration.<n>We conduct a systematic empirical study of state-of-the-art ASR architectures, encompassing encoder-decoder, transducer, and LLM-based paradigms, evaluated across batch, chunked, and streaming inference modes.<n>Our recommended configuration, the int4 k-quant variant, achieves 8.20% average streaming WER across eight standard benchmarks, running comfortably faster than real-time on CPU with 0.56 s algorithmic latency.
  arXiv  Detail & Related papers  (2026-04-16T00:04:32Z)
• Accelerating Video Generation Inference with Sequential-Parallel 3D Positional Encoding Using a Global Time Index [11.944339418621693]
  We implement system-level inference optimizations for a causal autoregressive video generation pipeline.<n>For five second 480P videos, a 1.58x speedup is achieved, thereby providing effective support for real-time interactive applications.
  arXiv  Detail & Related papers  (2026-03-02T10:18:18Z)
• IndexTTS 2.5 Technical Report [15.633832302970058]
  IndexTTS 2.5 significantly improves multilingual coverage, inference speed, and overall synthesis quality.<n>We replace the U-languageT-based backbone of the S2M module and establish practical design principles for zero-shot multilingual emotional TTS.<n>We apply GRPO in post-training of the T2S module, improving pronunciation accuracy and natrualness.
  arXiv  Detail & Related papers  (2026-01-07T12:58:16Z)
• StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation [65.90400162290057]
  Generative models are reshaping the live-streaming industry by redefining how content is created, styled, and delivered.<n>Recent advances in video diffusion have markedly improved temporal consistency and sampling efficiency for offline generation.<n>Live online streaming operates under strict service-level objectives (SLOs): time-to-first-frame must be minimal, and every frame must meet a per-frame deadline with low jitter.
  arXiv  Detail & Related papers  (2025-11-10T18:51:28Z)
• StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling [50.537794606598254]
  StreamMel is a pioneering single-stage streaming TTS framework that models continuous mel-spectrograms.<n>It enables low-latency, autoregressive synthesis while preserving high speaker similarity and naturalness.<n>It even achieves performance comparable to offline systems while supporting efficient real-time generation.
  arXiv  Detail & Related papers  (2025-06-14T16:53:39Z)
• Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech Synthesis [64.12708207721276]
  We introduce a novel pseudo-autoregressive (PAR) language modeling approach that unifies AR and NAR modeling.<n>Building on PAR, we propose PALLE, a two-stage TTS system that leverages PAR for initial generation followed by NAR refinement.<n>Experiments demonstrate that PALLE, trained on LibriTTS, outperforms state-of-the-art systems trained on large-scale data.
  arXiv  Detail & Related papers  (2025-04-14T16:03:21Z)
• SupertonicTTS: Towards Highly Efficient and Streamlined Text-to-Speech System [10.156665325783583]
  SupertonicTTS is a novel text-to-speech (TTS) system designed for efficient and streamlined speech synthesis.<n>We employ a low-dimensional latent space, temporal compression of latents, and ConvNeXt blocks to enable a lightweight architecture.<n> Experimental results demonstrate that SupertonicTTS delivers performance comparable to contemporary zero-shot TTS models with only 44M parameters.
  arXiv  Detail & Related papers  (2025-03-29T14:59:32Z)
• MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesis [56.25862714128288]
  This paper introduces textitMegaTTS 3, a zero-shot text-to-speech (TTS) system featuring an innovative sparse alignment algorithm.<n>Specifically, we provide sparse alignment boundaries to MegaTTS 3 to reduce the difficulty of alignment without limiting the search space.<n>Experiments demonstrate that MegaTTS 3 achieves state-of-the-art zero-shot TTS speech quality and supports highly flexible control over accent intensity.
  arXiv  Detail & Related papers  (2025-02-26T08:22:00Z)
• SimpleSpeech 2: Towards Simple and Efficient Text-to-Speech with Flow-based Scalar Latent Transformer Diffusion Models [64.40250409933752]
  We build upon our previous publication by implementing a simple and efficient non-autoregressive (NAR) TTS framework, termed SimpleSpeech 2. SimpleSpeech 2 effectively combines the strengths of both autoregressive (AR) and non-autoregressive (NAR) methods. We show a significant improvement in generation performance and generation speed compared to our previous work and other state-of-the-art (SOTA) large-scale TTS models.
  arXiv  Detail & Related papers  (2024-08-25T17:07:39Z)
• Efficient Incremental Text-to-Speech on GPUs [1.35346836945515]
  We present a highly efficient approach to perform real-time incremental TTS on GPUs with Instant Request Pooling and Module-wise Dynamic. Experimental results demonstrate that the proposed method is capable of producing high-quality speech with a first-chunk latency lower than 80ms under 100 QPS on a single NVIDIA A10 GPU.
  arXiv  Detail & Related papers  (2022-11-25T07:43:45Z)
• EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference [82.1584439276834]
  Transformer-based language models such as BERT provide significant accuracy improvement for a multitude of natural language processing (NLP) tasks. We present EdgeBERT, an in-depth algorithm- hardware co-design for latency-aware energy optimization for multi-task NLP.
  arXiv  Detail & Related papers  (2020-11-28T19:21:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.