Image-to-Video Diffusion: From Foundations to Open Frontiers

• URL: http://arxiv.org/abs/2605.17248v1
• Date: Sun, 17 May 2026 04:10:55 GMT
• Title: Image-to-Video Diffusion: From Foundations to Open Frontiers
• Abstract summary: Diffusion-based textitimage-to-video (I2V) generation has become a central direction in generative models.<n>This work treats diffusion I2V generation as a standalone subject.<n>It first reviews the task formulation, model architectures, datasets, and evaluation metrics, and then organizes existing methods through a taxonomy based on architecture and training paradigm.
• Score: 39.6216019326071
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion-based \textit{image-to-video} (I2V) generation has become a central direction in generative models by turning a reference image, with optional conditions, into a temporally coherent video. Compared with broader video generation settings, this task places stricter demands on content consistency, identity preservation, and motion coherence. Although the literature grows rapidly, existing works mostly discuss I2V generation within broader topics and still lack a dedicated taxonomy together with a systematic analysis centered on this field. This work addresses that gap by treating diffusion I2V generation as a standalone subject. It first reviews the task formulation, model architectures, datasets, and evaluation metrics, and then organizes existing methods through a taxonomy based on architecture and training paradigm. It further distills four core designs, namely condition encoding, temporal modeling, noise prior design, and spatial-temporal upsampling, and discusses representative application scenarios together with major open challenges.

Related papers

• UniVid: Pyramid Diffusion Model for High Quality Video Generation [10.349192062369088]
  Diffusion-based text-to-video generation (T2V) or image-to-video (I2V) generation have emerged as a prominent research focus.<n>We present a unified video generation model (UniVid) with hybrid conditions of the text prompt and reference image.
  arXiv  Detail & Related papers  (2026-03-14T03:51:16Z)
• SemanticGen: Video Generation in Semantic Space [60.49729308406981]
  State-of-the-art video generative models typically learn the distribution of video latents in the VAE space and map them to pixels using a VAE decoder.<n>We introduce SemanticGen, a novel solution to generate videos in the semantic space.<n>Our method is also effective and computationally efficient when extended to long video generation.
  arXiv  Detail & Related papers  (2025-12-23T18:59:56Z)
• BindWeave: Subject-Consistent Video Generation via Cross-Modal Integration [56.98981194478512]
  We propose a unified framework that handles a broad range of subject-to-video scenarios.<n>We introduce an MLLM-DiT framework in which a pretrained multimodal large language model performs deep cross-modal reasoning to ground entities.<n>Experiments on the OpenS2V benchmark demonstrate that our method achieves superior performance across subject consistency, naturalness, and text relevance in generated videos.
  arXiv  Detail & Related papers  (2025-10-01T02:41:11Z)
• StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation [117.13475564834458]
  We propose a new way of self-attention calculation, termed Consistent Self-Attention. To extend our method to long-range video generation, we introduce a novel semantic space temporal motion prediction module. By merging these two novel components, our framework, referred to as StoryDiffusion, can describe a text-based story with consistent images or videos.
  arXiv  Detail & Related papers  (2024-05-02T16:25:16Z)
• S2DM: Sector-Shaped Diffusion Models for Video Generation [2.0270353391739637]
  We propose a novel Sector-Shaped Diffusion Model (S2DM) for video generation. S2DM can generate a group of intrinsically related data sharing the same semantic and intrinsically related features. We show that, without additional training, our model integrated with another temporal conditions generative model can still achieve comparable performance with existing works.
  arXiv  Detail & Related papers  (2024-03-20T08:50:15Z)
• Hierarchical Spatio-temporal Decoupling for Text-to-Video Generation [49.298187741014345]
  Current methods intertwine spatial content and temporal dynamics together, leading to an increased complexity of text-to-video generation (T2V) We propose HiGen, a diffusion model-based method that improves performance by decoupling the spatial and temporal factors of videos from two perspectives.
  arXiv  Detail & Related papers  (2023-12-07T17:59:07Z)
• Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation [55.36617538438858]
  We propose a novel approach that strengthens the interaction between spatial and temporal perceptions. We curate a large-scale and open-source video dataset called HD-VG-130M.
  arXiv  Detail & Related papers  (2023-05-18T11:06:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.