FreeSwim: Revisiting Sliding-Window Attention Mechanisms for Training-Free Ultra-High-Resolution Video Generation

• URL: http://arxiv.org/abs/2511.14712v1
• Date: Tue, 18 Nov 2025 17:56:04 GMT
• Title: FreeSwim: Revisiting Sliding-Window Attention Mechanisms for Training-Free Ultra-High-Resolution Video Generation
• Abstract summary: We introduce a training-free approach that leverages video Diffusion Transformers pretrained at their native scale to synthesize higher resolution videos.<n>Our method delivers ultra-high-resolution videos with fine-grained visual details and high efficiency in a training-free paradigm.
• Score: 33.76347063890194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The quadratic time and memory complexity of the attention mechanism in modern Transformer based video generators makes end-to-end training for ultra high resolution videos prohibitively expensive. Motivated by this limitation, we introduce a training-free approach that leverages video Diffusion Transformers pretrained at their native scale to synthesize higher resolution videos without any additional training or adaptation. At the core of our method lies an inward sliding window attention mechanism, which originates from a key observation: maintaining each query token's training scale receptive field is crucial for preserving visual fidelity and detail. However, naive local window attention, unfortunately, often leads to repetitive content and exhibits a lack of global coherence in the generated results. To overcome this challenge, we devise a dual-path pipeline that backs up window attention with a novel cross-attention override strategy, enabling the semantic content produced by local attention to be guided by another branch with a full receptive field and, therefore, ensuring holistic consistency. Furthermore, to improve efficiency, we incorporate a cross-attention caching strategy for this branch to avoid the frequent computation of full 3D attention. Extensive experiments demonstrate that our method delivers ultra-high-resolution videos with fine-grained visual details and high efficiency in a training-free paradigm. Meanwhile, it achieves superior performance on VBench, even compared to training-based alternatives, with competitive or improved efficiency. Codes are available at: https://github.com/WillWu111/FreeSwim

Related papers

• Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity [57.83511884904928]
  We explore methods to accelerate training for autoregressive video generation models through empirical analyses.<n>Our results reveal that while training on fewer video frames significantly reduces training time, it also exacerbates error accumulation and introduces inconsistencies in the generated videos.<n>Inspired by Lipschitz continuity, we propose a Representation Continuity (ReCo) strategy to improve the consistency of generated videos.
  arXiv  Detail & Related papers  (2026-04-08T09:43:03Z)
• Semantic-Guided Unsupervised Video Summarization [5.891053607698674]
  We propose a novel Semantic-Guided Unsupervised Video Summarization method.<n>Specifically, we design a novel frame-level semantic alignment attention selector, which guides the Transformer-based generator within the adversarial framework to better reconstruct videos.<n>In addition, we adopt an incremental training strategy to progressively update the model components, effectively mitigating the instability of GAN training.
  arXiv  Detail & Related papers  (2026-01-21T08:53:29Z)
• Zero-Shot Video Translation and Editing with Frame Spatial-Temporal Correspondence [81.82643953694485]
  We present FRESCO, which integrates intra-frame correspondence with inter-frame correspondence to formulate a more robust spatial-temporal constraint.<n>Our method goes beyond attention guidance to explicitly optimize features, achieving high spatial-temporal consistency with the input video.<n>We verify FRESCO adaptations on two zero-shot tasks of video-to-video translation and text-guided video editing.
  arXiv  Detail & Related papers  (2025-12-03T15:51:11Z)
• UltraGen: High-Resolution Video Generation with Hierarchical Attention [62.99161115650818]
  UltraGen is a novel video generation framework that enables i) efficient and ii) end-to-end native high-resolution video synthesis.<n>We show that UltraGen can effectively scale pre-trained low-resolution video models to 1080P and even 4K resolution for the first time.
  arXiv  Detail & Related papers  (2025-10-21T16:23:21Z)
• MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learning [66.53533434848369]
  We propose a motion-guided self-learning framework that learns densely consistent representations.<n>We improve state-of-the-art by 1% to 6% on six image and video datasets and four evaluation benchmarks.
  arXiv  Detail & Related papers  (2025-06-10T11:20:32Z)
• ViaRL: Adaptive Temporal Grounding via Visual Iterated Amplification Reinforcement Learning [68.76048244253582]
  We introduce ViaRL, the first framework to leverage rule-based reinforcement learning (RL) for optimizing frame selection in video understanding.<n>ViaRL utilizes the answer accuracy of a downstream model as a reward signal to train a frame selector through trial-and-error.<n>ViaRL consistently delivers superior temporal grounding performance and robust generalization across diverse video understanding tasks.
  arXiv  Detail & Related papers  (2025-05-21T12:29:40Z)
• Ada-VE: Training-Free Consistent Video Editing Using Adaptive Motion Prior [13.595032265551184]
  Video-to-video synthesis poses significant challenges in maintaining character consistency, smooth temporal transitions, and preserving visual quality during fast motion. We propose an adaptive motion-guided cross-frame attention mechanism that selectively reduces redundant computations. This enables a greater number of cross-frame attentions over more frames within the same computational budget.
  arXiv  Detail & Related papers  (2024-06-07T12:12:25Z)
• Video Frame Interpolation Transformer [86.20646863821908]
  We propose a Transformer-based video framework that allows content-aware aggregation weights and considers long-range dependencies with the self-attention operations. To avoid the high computational cost of global self-attention, we introduce the concept of local attention into video. In addition, we develop a multi-scale frame scheme to fully realize the potential of Transformers.
  arXiv  Detail & Related papers  (2021-11-27T05:35:10Z)
• An Efficient Recurrent Adversarial Framework for Unsupervised Real-Time Video Enhancement [132.60976158877608]
  We propose an efficient adversarial video enhancement framework that learns directly from unpaired video examples. In particular, our framework introduces new recurrent cells that consist of interleaved local and global modules for implicit integration of spatial and temporal information. The proposed design allows our recurrent cells to efficiently propagate-temporal-information across frames and reduces the need for high complexity networks.
  arXiv  Detail & Related papers  (2020-12-24T00:03:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.