UltraViCo: Breaking Extrapolation Limits in Video Diffusion Transformers

• URL: http://arxiv.org/abs/2511.20123v1
• Date: Tue, 25 Nov 2025 09:44:10 GMT
• Title: UltraViCo: Breaking Extrapolation Limits in Video Diffusion Transformers
• Abstract summary: We propose a training-free, plug-and-play method that suppresses attention for tokens beyond the training window via a constant decay factor.<n>Remarkably, it improves Dynamic Degree and Imaging Quality by 233% and 40.5% over the previous best method at 4x extrapolation.
• Score: 42.379492244355454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite advances, video diffusion transformers still struggle to generalize beyond their training length, a challenge we term video length extrapolation. We identify two failure modes: model-specific periodic content repetition and a universal quality degradation. Prior works attempt to solve repetition via positional encodings, overlooking quality degradation and achieving only limited extrapolation. In this paper, we revisit this challenge from a more fundamental view: attention maps, which directly govern how context influences outputs. We identify that both failure modes arise from a unified cause: attention dispersion, where tokens beyond the training window dilute learned attention patterns. This leads to quality degradation and repetition emerges as a special case when this dispersion becomes structured into periodic attention patterns, induced by harmonic properties of positional encodings. Building on this insight, we propose UltraViCo, a training-free, plug-and-play method that suppresses attention for tokens beyond the training window via a constant decay factor. By jointly addressing both failure modes, we outperform a broad set of baselines largely across models and extrapolation ratios, pushing the extrapolation limit from 2x to 4x. Remarkably, it improves Dynamic Degree and Imaging Quality by 233% and 40.5% over the previous best method at 4x extrapolation. Furthermore, our method generalizes seamlessly to downstream tasks such as controllable video synthesis and editing.

Related papers

• FlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matching [68.01498128172214]
  We propose a novel but simple inference-time approach for long video generation that is architecture-agnostic and requires no additional training.<n>Our method generates long videos via overlapping sliding windows, where predicted clean samples from adjacent windows are blended via emphTweedie matching to enforce both textbfmanifold constraint and temporal consistency across overlap regions.<n>Our method generates videos several times longer than the native window length while outperforming both training-free and autoregressive baselines in temporal consistency and visual quality, and further extends to audio-video joint generation and text-to-3DGS without any
  arXiv  Detail & Related papers  (2026-05-20T08:55:37Z)
• Streaming Autoregressive Video Generation via Diagonal Distillation [50.13573884115673]
  Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity.<n>We propose Diagonal Distillation to exploit temporal information across both video chunks and denoising steps.<n>Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.
  arXiv  Detail & Related papers  (2026-03-10T10:45:24Z)
• All-in-One Video Restoration under Smoothly Evolving Unknown Weather Degradations [102.94052335735326]
  All-in-one image restoration aims to recover clean images from diverse unknown degradations using a single model.<n>Existing approaches primarily focus on frame-wise degradation variation, overlooking the temporal continuity that naturally exists in real-world degradation processes.<n>We introduce the Smoothly Evolving Unknown Degradations (SEUD) scenario, where both the active degradation set and degradation intensity change continuously over time.
  arXiv  Detail & Related papers  (2026-01-02T02:20:57Z)
• UltraImage: Rethinking Resolution Extrapolation in Image Diffusion Transformers [58.37965214608629]
  We present UltraImage, a principled framework that addresses content repetition and quality degradation.<n>Experiments show that UltraImage consistently outperforms prior methods on Qwen-Image and Flux (around 4K) across three generation scenarios.<n>UltraImage can generate images up to 6K*6K without low-resolution guidance from a training resolution of 1328p, demonstrating its extreme extrapolation capability.
  arXiv  Detail & Related papers  (2025-12-04T06:24:04Z)
• CtrlVDiff: Controllable Video Generation via Unified Multimodal Video Diffusion [62.04833878126661]
  We tackle the dual challenges of video understanding and controllable video generation within a unified diffusion framework.<n>We propose CtrlVDiff, a unified diffusion model trained with a Hybrid Modality Control Strategy (HMCS) that routes and fuses features from depth, normals, segmentation, edges, and graphics-based intrinsics (albedo, roughness, metallic)<n>Across understanding and generation benchmarks, CtrlVDiff delivers superior controllability and fidelity, enabling layer-wise edits (relighting, material adjustment, object insertion) and surpassing state-of-the-art baselines while remaining robust when some modalities are unavailable.
  arXiv  Detail & Related papers  (2025-11-26T07:27:11Z)
• FreeSwim: Revisiting Sliding-Window Attention Mechanisms for Training-Free Ultra-High-Resolution Video Generation [33.76347063890194]
  We introduce a training-free approach that leverages video Diffusion Transformers pretrained at their native scale to synthesize higher resolution videos.<n>Our method delivers ultra-high-resolution videos with fine-grained visual details and high efficiency in a training-free paradigm.
  arXiv  Detail & Related papers  (2025-11-18T17:56:04Z)
• FICGen: Frequency-Inspired Contextual Disentanglement for Layout-driven Degraded Image Generation [16.628211648386454]
  FICGen seeks to transfer frequency knowledge of degraded images into the latent diffusion space.<n>FICGen consistently surpasses existing L2I methods in terms of generative fidelity, alignment and downstream auxiliary trainability.
  arXiv  Detail & Related papers  (2025-09-01T04:00:22Z)
• Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion [67.94300151774085]
  We introduce Self Forcing, a novel training paradigm for autoregressive video diffusion models.<n>It addresses the longstanding issue of exposure bias, where models trained on ground-truth context must generate sequences conditioned on their own imperfect outputs.
  arXiv  Detail & Related papers  (2025-06-09T17:59:55Z)
• Re-ttention: Ultra Sparse Visual Generation via Attention Statistical Reshape [38.76559841681518]
  A huge bottleneck is the attention mechanism where complexity scales quadratically with resolution and video length.<n>Existing techniques fail to preserve visual quality at extremely high sparsity levels and might even incur non-negligible compute overheads.<n>We propose Re-ttention, which implements very high sparse attention for visual generation models.
  arXiv  Detail & Related papers  (2025-05-28T22:39:12Z)
• RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers [29.663251658875673]
  RIFLEx is a free lunch--achieving high-quality 2x extrapolation on state-of-the-art video diffusion transformers.<n>It enhances quality and enables 3x extrapolation by minimal fine-tuning without long videos.
  arXiv  Detail & Related papers  (2025-02-21T19:28:05Z)
• From Slow Bidirectional to Fast Autoregressive Video Diffusion Models [48.35054927704544]
  Current video diffusion models achieve impressive generation quality but struggle in interactive applications due to bidirectional attention dependencies.<n>We address this limitation by adapting a pretrained bidirectional diffusion transformer to an autoregressive transformer that generates frames on-the-fly.<n>Our model achieves a total score of 84.27 on the VBench-Long benchmark, surpassing all previous video generation models.
  arXiv  Detail & Related papers  (2024-12-10T18:59:50Z)
• Investigating Tradeoffs in Real-World Video Super-Resolution [90.81396836308085]
  Real-world video super-resolution (VSR) models are often trained with diverse degradations to improve generalizability. To alleviate the first tradeoff, we propose a degradation scheme that reduces up to 40% of training time without sacrificing performance. To facilitate fair comparisons, we propose the new VideoLQ dataset, which contains a large variety of real-world low-quality video sequences.
  arXiv  Detail & Related papers  (2021-11-24T18:58:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.