PARE: Pruning and Adaptive Routing for Efficient Video Generation

• URL: http://arxiv.org/abs/2605.27336v1
• Date: Tue, 26 May 2026 17:43:25 GMT
• Title: PARE: Pruning and Adaptive Routing for Efficient Video Generation
• Abstract summary: Video Diffusion Transformers (DiTs) generate high-quality videos but demand substantial compute due to wide blocks, deep architectures, and iterative sampling.<n>Recent methods reduce cost by compressing width, depth, or sampling steps, but typically commit to a fixed architecture that cannot adapt to individual inputs or denoising stages.<n>We propose PARE, which jointly compresses width and depth with structure-aware pruning and input-adaptive routing.
• Score: 71.54959622788608
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video Diffusion Transformers (DiTs) generate high-quality videos but demand substantial compute due to wide blocks, deep architectures, and iterative sampling. Recent methods reduce cost by compressing width, depth, or sampling steps, but typically commit to a fixed architecture that cannot adapt to individual inputs or denoising stages. We propose PARE (Pruning and Adaptive Routing for Efficient video generation), which jointly compresses width and depth with structure-aware pruning and input-adaptive routing. For width, we observe that attention heads specialize into spatial and temporal roles, and design importance scoring that accounts for this distinction to prevent motion-critical temporal heads from being pruned prematurely. For depth, we train a lightweight router conditioned on denoising timestep and visual content to dynamically select which blocks to execute at each step, enabling per-input compute adaptation rather than static block removal. A progressive pipeline first recovers width-pruned quality via distillation, then jointly optimizes the student and router to decouple the two learning objectives. Experiments on Wan2.1-14B for both image-to-video and text-to-video generation show that PARE substantially reduces per-step computation while preserving quality across VBench dimensions, and composes with step distillation for further acceleration.

Related papers

• Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models [6.202364388194408]
  Vision-language models excel at image and video understanding but suffer from high inference latency.<n>We propose a two-stage adaptive token pruning strategy specifically designed for video processing.<n>Our method is entirely post-hoc and requires no additional training or fine-tuning, while achieving strong empirical gains.
  arXiv  Detail & Related papers  (2026-08-04T04:32:48Z)
• Parallel Decoding Distillation for Fast Image and Video Generation [35.18845060233149]
  We introduce Parallel Decoding Distillation (PDD) for fast inference of diffusion and flow matching models.<n>PDD accelerates generation by predicting multiple denoising steps per network evaluation.<n>Our method achieves SOTA performance with 4-8 NFE on LTX-2.3 Text-to-Video/Audio, Wan 14B Text-to-Video, and Qwen-Image Text-to-Image.
  arXiv  Detail & Related papers  (2026-07-28T17:20:00Z)
• Kaleido: Algorithm-Hardware Co-Design for Video Diffusion Transformers by Exploiting Latent Space Correlations [47.69636045010864]
  Video diffusion transformers (vDiTs) generate high quality video but introduce extremely high compute cost.<n>Existing acceleration approaches largely inherit attention techniques from sparse models which fail to consider unique video data.<n>This paper presents Kale Kale, an algorithm hardware codesign that accelerates all operations in vDiTs by exploiting channel-wisetemporal correlations in latent space.<n> Evaluations across three mainstream vDiT models show that Kale Kale achieves 5.9x speedup and 16.0x energy savings over state of the art accelerators.
  arXiv  Detail & Related papers  (2026-07-15T12:31:19Z)
• EVATok: Adaptive Length Video Tokenization for Efficient Visual Autoregressive Generation [80.13014959623452]
  EVATok is a framework to produce $textbfE$fficient $textbfV$ideo $textbfA$daptive $textbfTok$enizers.<n>We show that EVATok achieves superior reconstruction and state-of-the-art class-to-video generation on UCF-101.
  arXiv  Detail & Related papers  (2026-03-12T17:59:59Z)
• Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generation [3.321460333625124]
  Diffusion Transformers (DiTs) achieve state-of-the-art results in text-to-image, text-to-video generation, and editing.<n> Static caching mitigates this by reusing features across fixed steps but fails to adapt to generation dynamics.<n>We propose Foresight, an adaptive layer-reuse technique that reduces computational redundancy across denoising steps while preserving baseline performance.
  arXiv  Detail & Related papers  (2025-05-31T00:52:17Z)
• VORTA: Efficient Video Diffusion via Routing Sparse Attention [54.84294780326206]
  VORTA is an acceleration framework with two novel components.<n>It achieves an end-to-end speedup $1.76times$ without loss of quality on VBench.<n>It can seamlessly integrate with various other acceleration methods, such as model caching and step distillation, reaching up to speedup $14.41times$ with negligible performance degradation.
  arXiv  Detail & Related papers  (2025-05-24T17:46:47Z)
• Training-free and Adaptive Sparse Attention for Efficient Long Video Generation [31.615453637053793]
  generating high-fidelity long videos with Diffusion Transformers (DiTs) is often hindered by significant latency.<n>We propose AdaSpa, the first Dynamic Pattern and Online Precise Search sparse attention method.<n>AdaSpa is implemented as an adaptive, plug-and-play solution and can be integrated seamlessly with existing DiTs.
  arXiv  Detail & Related papers  (2025-02-28T14:11:20Z)
• RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression [68.31184784672227]
  In modern applications such as autonomous driving, an overwhelming majority of videos serve as input for AI systems performing tasks.<n>It is therefore useful to optimize the encoder for a downstream task instead of for image quality.<n>Here, we address this challenge by controlling the Quantization Parameters (QPs) at the macro-block level to optimize the downstream task.
  arXiv  Detail & Related papers  (2025-01-21T15:36:08Z)
• SparseTem: Boosting the Efficiency of CNN-Based Video Encoders by Exploiting Temporal Continuity [19.900719882624028]
  We propose a memory-efficient scheduling method to eliminate memory overhead and an online adjustment mechanism to minimize accuracy degradation.<n>SparseTem achieves speedup of 1.79x for EfficientDet and 4.72x for CRNN, with minimal accuracy drop and no additional memory overhead.
  arXiv  Detail & Related papers  (2024-10-28T07:13:25Z)
• ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler [53.98558445900626]
  Current image-to-video diffusion models, while powerful in generating videos from a single frame, need adaptation for two-frame conditioned generation.<n>We introduce a novel, bidirectional sampling strategy to address these off-manifold issues without requiring extensive re-noising or fine-tuning.<n>Our method employs sequential sampling along both forward and backward paths, conditioned on the start and end frames, respectively, ensuring more coherent and on-manifold generation of intermediate frames.
  arXiv  Detail & Related papers  (2024-10-08T03:01:54Z)
• Low-Fidelity End-to-End Video Encoder Pre-training for Temporal Action Localization [96.73647162960842]
  TAL is a fundamental yet challenging task in video understanding. Existing TAL methods rely on pre-training a video encoder through action classification supervision. We introduce a novel low-fidelity end-to-end (LoFi) video encoder pre-training method.
  arXiv  Detail & Related papers  (2021-03-28T22:18:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.