Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation

• URL: http://arxiv.org/abs/2606.23743v2
• Date: Wed, 24 Jun 2026 14:59:15 GMT
• Title: Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation
• Abstract summary: We present Sol Video Inference, an agentic, native, training-free acceleration framework for video diffusion models.<n>It organizes five broadly applicable techniques, cache, sparse attention, token pruning, quantization, and kernel fusion, into an agentic acceleration stack.<n>It achieves more than 2x end-to-end acceleration while maintaining near-lossless VBench quality.
• Score: 50.504140880964336
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost. Although many acceleration methods have been proposed, a central challenge is that the most effective acceleration strategy is highly instance-specific: a recipe that works well for one combination of model, hardware, and inference configuration often does not transfer to another. Different models vary in architecture, numerical sensitivity, and attention concentration patterns. Inference settings differ in spatial and temporal resolution and video duration, while hardware platforms differ in memory hierarchy, supported numerical formats, and kernel throughput. These factors create a large tuning space, making manual performance engineering costly. We present Sol Video Inference Engine, an agentic, native, training-free acceleration framework for video diffusion models. It organizes five broadly applicable techniques, cache, sparse attention, token pruning, quantization, and kernel fusion, into an agentic acceleration stack for instance-specific optimization. For a concrete deployment target defined by a model, hardware platform, and serving configuration, parallel skill agents optimize the implementation of each technique, an agent integrator composes them into a global acceleration stack, and a human validator provides feedback on generation quality. We instantiate this workflow on three video models with different sizes and architectures: 64B Cosmos3-Super, 22B LTX-2.3, and 2B SANA-Video. With little human effort, the full stack achieves more than 2x end-to-end acceleration while maintaining near-lossless VBench quality, demonstrating the effectiveness of the agent framework for video diffusion acceleration.

Related papers

• VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding [82.55178663351913]
  VideoChat3 is a fully open, efficient, and generalist video-centric MLLM.<n>For efficiency, we introduce Inflated 3D Vision Transformer (ID-ViT) and Adaptive Frame Resolution for Streaming Video Perception.<n>For effectiveness, we develop a data synthesis pipeline that curates three diverse, high-quality training datasets.
  arXiv  Detail & Related papers  (2026-07-16T12:47:59Z)
• LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing [28.426681930851604]
  We present LoomVideo, a highly efficient 5B- parameter unified architecture for both video generation and editing.<n>By scaling and directly adding the clean source video latent to the noised target latent, this elegant design eliminates the need for token concatenation.<n>Our compact 5B model achieves state-of-the-art or highly competitive performance across comprehensive benchmarks.
  arXiv  Detail & Related papers  (2026-06-04T11:35:26Z)
• RainFusion2.0: Temporal-Spatial Awareness and Hardware-Efficient Block-wise Sparse Attention [10.041502708600865]
  RainFusion2.0 is an online adaptive, hardware-efficient, and low-overhead sparse attention mechanism to accelerate video and image generative models.<n>It can achieve 80% sparsity while achieving an end-to-end speedup of 1.51.8x without compromising video quality.
  arXiv  Detail & Related papers  (2025-12-30T08:55:20Z)
• Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Caching [57.7533917467934]
  EasyCache is a training-free acceleration framework for video diffusion models.<n>We conduct comprehensive studies on various large-scale video generation models, including OpenSora, Wan2.1, and HunyuanVideo.<n>Our method achieves leading acceleration performance, reducing inference time by up to 2.1-3.3$times$ compared to the original baselines.
  arXiv  Detail & Related papers  (2025-07-03T17:59:54Z)
• MambaMia: A State-Space-Model-Based Compression for Efficient Video Understanding in Large Multimodal Models [33.89483627891117]
  We propose an efficient framework to compress multiple video-frame features before feeding them into large multimodal models.<n>Our framework emphasizes resource-conscious efficiency, making it practical for real-world deployments.
  arXiv  Detail & Related papers  (2025-06-16T14:49:49Z)
• Astraea: A Token-wise Acceleration Framework for Video Diffusion Transformers [29.130090574300635]
  Video diffusion transformers (vDiTs) have made tremendous progress in text-to-video generation, but their compute demands pose a major challenge for practical deployment.<n>We introduce Astraea, a framework that searches for near-optimal configurations for vDiT-based video generation under a performance target.
  arXiv  Detail & Related papers  (2025-06-05T14:41:38Z)
• QuantCache: Adaptive Importance-Guided Quantization with Hierarchical Latent and Layer Caching for Video Generation [84.91431271257437]
  Diffusion Transformers (DiTs) have emerged as a dominant architecture in video generation.<n>DiTs come with significant drawbacks, including increased computational and memory costs.<n>We propose QuantCache, a novel training-free inference acceleration framework.
  arXiv  Detail & Related papers  (2025-03-09T10:31:51Z)
• Adaptive Caching for Faster Video Generation with Diffusion Transformers [52.73348147077075]
  Diffusion Transformers (DiTs) rely on larger models and heavier attention mechanisms, resulting in slower inference speeds. We introduce a training-free method to accelerate video DiTs, termed Adaptive Caching (AdaCache) We also introduce a Motion Regularization (MoReg) scheme to utilize video information within AdaCache, controlling the compute allocation based on motion content.
  arXiv  Detail & Related papers  (2024-11-04T18:59:44Z)
• Fast and Memory-Efficient Video Diffusion Using Streamlined Inference [41.505829393818274]
  Current video diffusion models exhibit demanding computational requirements and high peak memory usage. We present Streamlined Inference, which leverages the temporal and spatial properties of video diffusion models. Our approach significantly reduces peak memory and computational overhead, making it feasible to generate high-quality videos on a single consumer GPU.
  arXiv  Detail & Related papers  (2024-11-02T07:52:18Z)
• Hierarchical Patch Diffusion Models for High-Resolution Video Generation [50.42746357450949]
  We develop deep context fusion, which propagates context information from low-scale to high-scale patches in a hierarchical manner. We also propose adaptive computation, which allocates more network capacity and computation towards coarse image details. The resulting model sets a new state-of-the-art FVD score of 66.32 and Inception Score of 87.68 in class-conditional video generation.
  arXiv  Detail & Related papers  (2024-06-12T01:12:53Z)
• A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames [57.758863967770594]
  We build on the common paradigm of transferring large-scale, image--text models to video via shallow temporal fusion.<n>We expose two limitations to the approach: (1) decreased spatial capabilities, likely due to poor video--language alignment in standard video datasets, and (2) higher memory consumption, bottlenecking the number of frames that can be processed.
  arXiv  Detail & Related papers  (2023-12-12T16:10:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.