TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation

• URL: http://arxiv.org/abs/2608.24674v1
• Date: Tue, 25 Aug 2026 15:09:02 GMT
• Title: TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation
• Abstract summary: TurboT2VA is a distillation and inference framework for joint text-to-video-audio generation.<n>On LTX-2, four-step distillation reduces generator latency from 50.52s to 2.51s at the standard evaluation resolution of 512$times$768.<n>The complete stack reduces generator latency from 318.74s to 5.83s on one NVIDIA H20, achieving a 54.67$times$ generator-only speedup.
• Score: 105.14745366152725
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal computation of large models make inference prohibitively expensive. We present TurboT2VA, a distillation and inference framework for accelerating a 19B-parameter joint video-audio model. Large-scale T2VA distillation is challenged by modality-imbalanced optimization, the difficulty of continuous-time consistency training at scale, and the quality--diversity trade-off. TurboT2VA addresses these issues with per-modality normalization and a progressive curriculum comprising discrete consistency warm-up, continuous consistency refinement, and joint consistency--distribution matching. The curriculum first establishes a stable, diverse generation trajectory and only then introduces distribution-level refinement. On LTX-2, four-step distillation reduces generator latency from 50.52s to 2.51s at the standard evaluation resolution of 512$\times$768, achieving a 20.1$\times$ speedup while maintaining strong visual quality, audio fidelity, diversity, and video-audio synchronization. We further develop an architecture-aware inference stack that combines guarded W8A8 and fused operators, padded-text compaction, and modality-aware sparse attention while preserving dense cross-modal and text-conditioning paths. Under the high-resolution deployment setting at 1024$\times$1792, the complete stack reduces generator latency from 318.74s to 5.83s on one NVIDIA H20, achieving a 54.67$\times$ generator-only speedup. Inference code and generation demos are available at https://github.com/thu-ml/TurboDiffusion/tree/main/turbot2va.

Related papers

• DiffusionGemma Technical Report [72.10570774996219]
  We introduce DiffusionGemma, a language model that uses discrete diffusion to generate text at exceptionally high speed.<n>Rather than decoding one token at a time, DiffusionGemma iteratively refines blocks of 256 tokens in parallel.<n>Our compute-efficient two-stage training pipeline uses fewer than 10% of the starting AR model's total training token budget.
  arXiv  Detail & Related papers  (2026-07-31T16:11:46Z)
• Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference [5.685908474026849]
  High-quality automatic speech recognition (ASR) on edge devices requires models that jointly optimize accuracy, latency, and memory footprint while operating entirely on CPU without GPU acceleration.<n>We conduct a systematic empirical study of state-of-the-art ASR architectures, encompassing encoder-decoder, transducer, and LLM-based paradigms, evaluated across batch, chunked, and streaming inference modes.<n>Our recommended configuration, the int4 k-quant variant, achieves 8.20% average streaming WER across eight standard benchmarks, running comfortably faster than real-time on CPU with 0.56 s algorithmic latency.
  arXiv  Detail & Related papers  (2026-04-16T00:04:32Z)
• StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation [65.90400162290057]
  Generative models are reshaping the live-streaming industry by redefining how content is created, styled, and delivered.<n>Recent advances in video diffusion have markedly improved temporal consistency and sampling efficiency for offline generation.<n>Live online streaming operates under strict service-level objectives (SLOs): time-to-first-frame must be minimal, and every frame must meet a per-frame deadline with low jitter.
  arXiv  Detail & Related papers  (2025-11-10T18:51:28Z)
• AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising [49.785626309848276]
  AsyncDiff is a universal and plug-and-play acceleration scheme that enables model parallelism across multiple devices. For the Stable Diffusion v2.1, AsyncDiff achieves a 2.7x speedup with negligible degradation and a 4.0x speedup with only a slight reduction of 0.38 in CLIP Score. Our experiments also demonstrate that AsyncDiff can be readily applied to video diffusion models with encouraging performances.
  arXiv  Detail & Related papers  (2024-06-11T03:09:37Z)
• T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback [111.40967379458752]
  We introduce T2V-Turbo, which integrates feedback from a mixture of differentiable reward models into the consistency distillation process of a pre-trained T2V model. Remarkably, the 4-step generations from our T2V-Turbo achieve the highest total score on VBench, even surpassing Gen-2 and Pika.
  arXiv  Detail & Related papers  (2024-05-29T04:26:17Z)
• StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation [52.56469577812338]
  We introduce StreamDiffusion, a real-time diffusion pipeline for interactive image generation.<n>Existing diffusion models are adept at creating images from text or image prompts, yet they often fall short in real-time interaction.<n>We present a novel approach that transforms the original sequential denoising into the denoising process.
  arXiv  Detail & Related papers  (2023-12-19T18:18:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.