Edit2Interp: Adapting Image Foundation Models from Spatial Editing to Video Frame Interpolation with Few-Shot Learning

• URL: http://arxiv.org/abs/2603.15003v1
• Date: Mon, 16 Mar 2026 09:07:25 GMT
• Title: Edit2Interp: Adapting Image Foundation Models from Spatial Editing to Video Frame Interpolation with Few-Shot Learning
• Abstract summary: We show that a large image editing model (Qwen-Image-Edit) can be adapted for Video Frame Interpolation (VFI) using only 64-256 training samples via Low-Rank Adaptation (LoRA)<n>Our work establishes that foundation image editing models possess untapped potential for temporal tasks, offering a data-efficient pathway for video synthesis in resource-constrained scenarios.
• Score: 5.6616478694920405
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pre-trained image editing models exhibit strong spatial reasoning and object-aware transformation capabilities acquired from billions of image-text pairs, yet they possess no explicit temporal modeling. This paper demonstrates that these spatial priors can be repurposed to unlock temporal synthesis capabilities through minimal adaptation - without introducing any video-specific architecture or motion estimation modules. We show that a large image editing model (Qwen-Image-Edit), originally designed solely for static instruction-based edits, can be adapted for Video Frame Interpolation (VFI) using only 64-256 training samples via Low-Rank Adaptation (LoRA). Our core contribution is revealing that the model's inherent understanding of "how objects transform" in static scenes contains latent temporal reasoning that can be activated through few-shot fine-tuning. While the baseline model completely fails at producing coherent intermediate frames, our parameter-efficient adaptation successfully unlocks its interpolation capability. Rather than competing with task-specific VFI methods trained from scratch on massive datasets, our work establishes that foundation image editing models possess untapped potential for temporal tasks, offering a data-efficient pathway for video synthesis in resource-constrained scenarios. This bridges the gap between image manipulation and video understanding, suggesting that spatial and temporal reasoning may be more intertwined in foundation models than previously recognized

Related papers

• Imagine Before You Draw: Visual Prompt Engineering for Image Generation [67.81347924426714]
  We propose Visual Prompt Engineering (VPE), which can be seamlessly integrated into internal frameworks.<n>We validate VPE across class-conditional generation, text-to-image generation, and image editing.<n>Results show that VPE can accelerate convergence, raise quality ceilings, and through internal integration, achieve substantially better editing preservation.
  arXiv  Detail & Related papers  (2026-06-03T05:01:36Z)
• ImVideoEdit: Image-learning Video Editing via 2D Spatial Difference Attention Blocks [46.442660004277734]
  We propose ImVideoEdit, an efficient framework that learns video editing capabilities entirely from image pairs.<n>By freezing the pre-trained 3D attention modules and treating images as single-frame videos, we decouple the 2D spatial learning process to help preserve the original temporal dynamics.<n>Despite training on only 13K image pairs for 5 epochs with exceptionally low computational overhead, ImVideoEdit achieves editing fidelity and temporal consistency comparable to larger models trained on extensive video datasets.
  arXiv  Detail & Related papers  (2026-04-09T08:22:09Z)
• From Static to Dynamic: Exploring Self-supervised Image-to-Video Representation Transfer Learning [79.76998842796515]
  Fine-tuning heavy modules may compromise inter-video semantic separability.<n>We propose the Consistency-Separability Trade-off Transfer Learning framework.<n>Experiments on eight image-pretrained models demonstrate consistent improvements across multiple levels of video tasks.
  arXiv  Detail & Related papers  (2026-03-27T16:56:50Z)
• SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing [76.349958946335]
  We present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling.<n>First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames.<n>Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks.
  arXiv  Detail & Related papers  (2026-03-19T17:59:51Z)
• Video4Edit: Viewing Image Editing as a Degenerate Temporal Process [24.8621496006791]
  multimodal foundation models have propelled instruction-driven image generation and editing into a genuinely cross-modal, cooperative regime.<n>We revisit this challenge through the lens of temporal modeling.<n>This perspective allows us to transfer single-frame evolution priors from video pre-training, enabling a highly data-efficient fine-tuning regime.
  arXiv  Detail & Related papers  (2025-11-22T17:30:55Z)
• VALA: Learning Latent Anchors for Training-Free and Temporally Consistent [29.516179213427694]
  We propose VALA, a variational alignment module that adaptively selects key frames and compresses their latent features into semantic anchors for consistent video editing.<n>Our method can be fully integrated into training-free text-to-image based video editing models.
  arXiv  Detail & Related papers  (2025-10-27T03:44:11Z)
• Subject-driven Video Generation via Disentangled Identity and Motion [52.54835936914813]
  We propose to train a subject-driven customized video generation model through decoupling the subject-specific learning from temporal dynamics in zero-shot without additional tuning.<n>Our method achieves strong subject consistency and scalability, outperforming existing video customization models in zero-shot settings.
  arXiv  Detail & Related papers  (2025-04-23T06:48:31Z)
• EIDT-V: Exploiting Intersections in Diffusion Trajectories for Model-Agnostic, Zero-Shot, Training-Free Text-to-Video Generation [26.888320234592978]
  Zero-shot, training-free, image-based text-to-video generation is an emerging area that aims to generate videos using existing image-based diffusion models.<n>We provide a model-agnostic approach, using intersections in diffusion trajectories, working only with latent values.<n>An in-context trained LLM is used to generate coherent frame-wise prompts; another is used to identify differences between frames.<n>Our approach results in state-of-the-art performance while being more flexible when working with diverse image-generation models.
  arXiv  Detail & Related papers  (2025-04-09T13:11:09Z)
• Multi-subject Open-set Personalization in Video Generation [110.02124633005516]
  We present Video Alchemist $-$ a video model with built-in multi-subject, open-set personalization capabilities.<n>Our model is built on a new Diffusion Transformer module that fuses each conditional reference image and its corresponding subject-level text prompt.<n>Our method significantly outperforms existing personalization methods in both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2025-01-10T18:59:54Z)
• VASE: Object-Centric Appearance and Shape Manipulation of Real Videos [108.60416277357712]
  In this work, we introduce a framework that is object-centric and is designed to control both the object's appearance and, notably, to execute precise and explicit structural modifications on the object. We build our framework on a pre-trained image-conditioned diffusion model, integrate layers to handle the temporal dimension, and propose training strategies and architectural modifications to enable shape control. We evaluate our method on the image-driven video editing task showing similar performance to the state-of-the-art, and showcasing novel shape-editing capabilities.
  arXiv  Detail & Related papers  (2024-01-04T18:59:24Z)
• Fine-tuned CLIP Models are Efficient Video Learners [54.96069171726668]
  Large-scale multi-modal training with image-text pairs imparts strong generalization to CLIP model. Video Fine-tuned CLIP (ViFi-CLIP) baseline is generally sufficient to bridge the domain gap from images to videos.
  arXiv  Detail & Related papers  (2022-12-06T18:59:58Z)
• StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN [70.31913835035206]
  We present a novel approach to the video synthesis problem that helps to greatly improve visual quality. We make use of a pre-trained StyleGAN network, the latent space of which allows control over the appearance of the objects it was trained for. Our temporal architecture is then trained not on sequences of RGB frames, but on sequences of StyleGAN latent codes.
  arXiv  Detail & Related papers  (2021-07-15T09:58:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.