SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation

• URL: http://arxiv.org/abs/2607.21553v1
• Date: Thu, 23 Jul 2026 17:36:05 GMT
• Title: SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation
• Abstract summary: We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales.<n>It matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention.<n>With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100.
• Score: 62.15917923814872
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.

Related papers

• SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer [45.09089555026137]
  We introduce SANA-WM, an efficient open-source world model trained for one-minute generation, high-fidelity, 720p, minute-scale videos with precise camera control.<n>SANA-WM achieves visual quality comparable to large-scale industrial baselines such as LingBot-World and HY-WorldPlay.
  arXiv  Detail & Related papers  (2026-05-14T17:58:03Z)
• CubeComposer: Spatio-Temporal Autoregressive 4K 360° Video Generation from Perspective Video [86.80231588752957]
  We introduce a novel Cube-temporal autoregressive diffusion model that generates 4K-resolution 360 videos.<n>By decomposing videos into cubemap representations with six faces, CubeComposer autogressively synthesizes content in a well-planned order.<n> experiments on benchmark datasets demonstrate that CubeComposer outperforms state-of-the-art methods in native resolution and visual quality.
  arXiv  Detail & Related papers  (2026-03-04T17:06:56Z)
• ReHyAt: Recurrent Hybrid Attention for Video Diffusion Transformers [10.830662834634879]
  ReHyAt is a hybrid attention mechanism that combines the fidelity of softmax attention with the efficiency of linear attention.<n>Our experiments demonstrate that ReHyAt achieves state-of-the-art video quality while reducing attention cost from quadratic to linear.
  arXiv  Detail & Related papers  (2026-01-07T19:26:30Z)
• DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder [55.26098043655325]
  DC-VideoGen can be applied to any pre-trained video diffusion model.<n>It can be adapted to a deep compression latent space with lightweight fine-tuning.
  arXiv  Detail & Related papers  (2025-09-29T17:59:31Z)
• Attention Surgery: An Efficient Recipe to Linearize Your Video Diffusion Transformer [13.545000689565732]
  Transformer-based video diffusion models (VDMs) deliver state-of-the-art video generation quality but are constrained by the quadratic cost of self-attention.<n>We introduce Attention Surgery, an efficient framework for linearizing or hybridizing attention in pretrained VDMs without training from scratch.
  arXiv  Detail & Related papers  (2025-09-29T15:09:51Z)
• SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer [116.17385614259574]
  We introduce SANA-Video, a small diffusion model that can efficiently generate videos up to 720x1280 resolution and minute-length duration.<n>Two core designs ensure our efficient, effective and long video generation.<n>Given its low cost, SANA-Video achieves competitive performance compared to modern state-of-the-art small diffusion models.
  arXiv  Detail & Related papers  (2025-09-29T12:28:09Z)
• BLADE: Block-Sparse Attention Meets Step Distillation for Efficient Video Generation [27.57431718095974]
  We present BLADE, a data-free joint training framework for video inference.<n>We show remarkable efficiency gains across different scales.<n>On models such as CogVideoX-5B with short video sequence lengths, our framework delivers a robust 8.89x speedup.
  arXiv  Detail & Related papers  (2025-08-14T15:58:59Z)
• An LMM for Efficient Video Understanding via Reinforced Compression of Video Cubes [85.00111442236499]
  This paper presents textbfQuicksviewer, an LMM with new perceiving paradigm that partitions a video of nontemporal density into varying cubes using Gumbel Softmax.<n>We train the model from a language backbone through three progressive stages, each incorporating lengthy videos on average of 420s/1fps thanks to the perceiving efficiency.<n>With only 0.8M total video-text samples for training, our model outperforms the direct baseline employing a fixed partitioning strategy by a maximum of 8.72 in accuracy.
  arXiv  Detail & Related papers  (2025-04-21T17:57:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.