SwiftI2V: Efficient High-Resolution Image-to-Video Generation via Conditional Segment-wise Generation

• URL: http://arxiv.org/abs/2605.06356v2
• Date: Sat, 09 May 2026 12:33:33 GMT
• Title: SwiftI2V: Efficient High-Resolution Image-to-Video Generation via Conditional Segment-wise Generation
• Abstract summary: High-resolution image-to-video (I2V) generation aims to synthesize realistic temporal dynamics while preserving fine-grained appearance details of the input image.<n>Existing solutions suffer from various weaknesses: 1) end-to-end models are often prohibitively expensive in memory and latency; 2) cascading low-resolution generation with a generic video super-resolution tends to hallucinate details and drift from input-specific local structures.<n>We propose SwiftI2V, an efficient framework tailored for high-resolution I2V.
• Score: 17.915677925345722
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-resolution image-to-video (I2V) generation aims to synthesize realistic temporal dynamics while preserving fine-grained appearance details of the input image. At 2K resolution, it becomes extremely challenging, and existing solutions suffer from various weaknesses: 1) end-to-end models are often prohibitively expensive in memory and latency; 2) cascading low-resolution generation with a generic video super-resolution tends to hallucinate details and drift from input-specific local structures, since the super-resolution stage is not explicitly conditioned on the input image. To this end, we propose SwiftI2V, an efficient framework tailored for high-resolution I2V. Following the widely used two-stage design, it addresses the efficiency--fidelity dilemma by first generating a low-resolution motion reference to reduce token costs and ease the modeling burden, then performing a strongly image-conditioned 2K synthesis guided by the motion to recover input-faithful details with controlled overhead. Specifically, to make generation more scalable, SwiftI2V introduces Conditional Segment-wise Generation (CSG) to synthesize videos segment-by-segment with a bounded per-step token budget, and adopts bidirectional contextual interaction within each segment to improve cross-segment coherence and input fidelity. On VBench-I2V at 2K resolution, SwiftI2V achieves performance comparable to end-to-end baselines while reducing total GPU-time by 202x. Particularly, it enables practical 2K I2V generation on a single datacenter GPU (e.g., H800) or consumer GPU (e.g., RTX 4090).

Related papers

• LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models [14.33633321388011]
  LongE2V is a novel approach that leverages pre-trained video diffusion priors to handle event-based video reconstruction.<n>By fine-tuning agressive video models, our approach mitigates high data efficiency and superior perceptual quality.<n>Experiments on real-world benchmarks demonstrate that LongE2V outperforms state-of-the-art methods across all three tasks.
  arXiv  Detail & Related papers  (2026-07-09T17:59:52Z)
• HiReFF: High-Resolution Feedforward Human Reconstruction from Uncalibrated Sparse-View Video [58.85548854304172]
  HiReFF is a feed-forward method for 2K-resolution 360 human video reconstruction from uncalibrated sparse-view videos.<n>Our framework decomposes the problem into two key tasks: 3D Gaussian reconstruction from sparse-view videos and computationally efficient high-resolution synthesis.<n> Experiments demonstrate that HiReFF significantly outperforms existing methods in high-resolution streaming video reconstruction.
  arXiv  Detail & Related papers  (2026-06-28T11:01:36Z)
• Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions [69.01904860240938]
  Ultra Flash is a cascaded streaming framework capable of real-time high-resolution video generation.<n>It produces ultra-high-resolution streaming video while maintaining state-of-the-art visual quality and superior efficiency.
  arXiv  Detail & Related papers  (2026-06-08T07:45:03Z)
• ViBe: Ultra-High-Resolution Video Synthesis Born from Pure Images [30.646542711556787]
  Transformer-based video diffusion models rely on 3D attention over spatial and temporal tokens.<n>We propose a pure image adaptation framework that upgrades a video Diffusion Transformer pre-trained at its native scale to synthesize higher-resolution videos.
  arXiv  Detail & Related papers  (2026-03-24T15:27:22Z)
• UltraGen: High-Resolution Video Generation with Hierarchical Attention [62.99161115650818]
  UltraGen is a novel video generation framework that enables i) efficient and ii) end-to-end native high-resolution video synthesis.<n>We show that UltraGen can effectively scale pre-trained low-resolution video models to 1080P and even 4K resolution for the first time.
  arXiv  Detail & Related papers  (2025-10-21T16:23:21Z)
• Turbo2K: Towards Ultra-Efficient and High-Quality 2K Video Synthesis [50.77548592888096]
  Demand for 2K video synthesis is rising with increasing consumer expectations for ultra-clear visuals.<n>Turbo2K is an efficient framework for generating detail-rich 2K videos.
  arXiv  Detail & Related papers  (2025-04-20T03:30:59Z)
• STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolution [42.859188375578604]
  Image diffusion models have been adapted for real-world video superresolution to tackle over-smoothing issues in GAN-based methods.<n>These models struggle to maintain temporal consistency, as they are trained on static images.<n>We introduce a novel approach that leverages T2V models for real-world video super-resolution, achieving realistic spatial details and robust temporal consistency.
  arXiv  Detail & Related papers  (2025-01-06T12:36:21Z)
• LTX-Video: Realtime Video Latent Diffusion [4.7789714048042775]
  LTX-Video is a transformer-based latent diffusion model.<n>It seamlessly integrates the Video-VAE and the denoising transformer.<n>It achieves faster-than-real-time generation, producing 5 seconds of 24 fps video 768 atx512 resolution in just 2 seconds on an Nvidia H100 GPU.
  arXiv  Detail & Related papers  (2024-12-30T19:00:25Z)
• ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler [53.98558445900626]
  Current image-to-video diffusion models, while powerful in generating videos from a single frame, need adaptation for two-frame conditioned generation.<n>We introduce a novel, bidirectional sampling strategy to address these off-manifold issues without requiring extensive re-noising or fine-tuning.<n>Our method employs sequential sampling along both forward and backward paths, conditioned on the start and end frames, respectively, ensuring more coherent and on-manifold generation of intermediate frames.
  arXiv  Detail & Related papers  (2024-10-08T03:01:54Z)
• I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models [80.32562822058924]
  Text-guided image-to-video (I2V) generation aims to generate a coherent video that preserves the identity of the input image. I2V-Adapter adeptly propagates the unnoised input image to subsequent noised frames through a cross-frame attention mechanism. Our experimental results demonstrate that I2V-Adapter is capable of producing high-quality videos.
  arXiv  Detail & Related papers  (2023-12-27T19:11:50Z)
• Low-Resolution Self-Attention for Semantic Segmentation [93.30597515880079]
  We introduce the Low-Resolution Self-Attention (LRSA) mechanism to capture global context at a significantly reduced computational cost.<n>Our approach involves computing self-attention in a fixed low-resolution space regardless of the input image's resolution.<n>We demonstrate the effectiveness of our LRSA approach by building the LRFormer, a vision transformer with an encoder-decoder structure.
  arXiv  Detail & Related papers  (2023-10-08T06:10:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.