ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformer

• URL: http://arxiv.org/abs/2603.15478v1
• Date: Mon, 16 Mar 2026 16:10:46 GMT
• Title: ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformer
• Abstract summary: We propose a video-free tuning framework termed ViFeEdit for video diffusion transformers.<n>ViFeEdit achieves versatile video generation and editing, adapted solely with 2D images.<n>Our method delivers promising results of controllable video generation and editing with only minimal training on 2D image data.
• Score: 74.61793196579036
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion Transformers (DiTs) have demonstrated remarkable scalability and quality in image and video generation, prompting growing interest in extending them to controllable generation and editing tasks. However, compared to the image counterparts, progress in video control and editing remains limited, mainly due to the scarcity of paired video data and the high computational cost of training video diffusion models. To address this issue, in this paper, we propose a video-free tuning framework termed ViFeEdit for video diffusion transformers. Without requiring any forms of video training data, ViFeEdit achieves versatile video generation and editing, adapted solely with 2D images. At the core of our approach is an architectural reparameterization that decouples spatial independence from the full 3D attention in modern video diffusion transformers, which enables visually faithful editing while maintaining temporal consistency with only minimal additional parameters. Moreover, this design operates in a dual-path pipeline with separate timestep embeddings for noise scheduling, exhibiting strong adaptability to diverse conditioning signals. Extensive experiments demonstrate that our method delivers promising results of controllable video generation and editing with only minimal training on 2D image data. Codes are available https://github.com/Lexie-YU/ViFeEdit.

Related papers

• Instruction-Based Video Editing by Repurposing an Image Editing Model [55.9214783600876]
  We show that a strong instruction-based image editing model can edit videos by operating directly on video-VAE latents.<n>Our results suggest that, despite the large investment in training video latent spaces, per-frame video latents remain close enough to the image domain that mature image editing priors transfer with minimal adaptation.
  arXiv  Detail & Related papers  (2026-08-14T18:01:29Z)
• OSVE: One Step Video Editing with One Step Diffusion Models [17.564785522861477]
  We present OSVE, the first framework to successfully adapt one-step Text-to-Image (T2I) models for high-quality video editing.<n>We address the core challenges of inversion, editability, and temporal consistency.<n>This breakthrough paves the way for practical, real-time video editing applications.
  arXiv  Detail & Related papers  (2026-07-22T08:29:30Z)
• EasyV2V: A High-quality Instruction-based Video Editing Framework [108.78294392167017]
  captionemphEasyV2V is a framework for instruction-based video editing.<n>EasyV2V works with flexible inputs, e.g., video+text, video+mask+reference+, and state-of-the-art video editing results.
  arXiv  Detail & Related papers  (2025-12-18T18:59:57Z)
• CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer [51.805505207941934]
  We present CogVideoX, a large-scale text-to-video generation model based on diffusion transformer.<n>It can generate 10-second continuous videos aligned with text prompt, with a frame rate of 16 fps and resolution of 768 * 1360 pixels.
  arXiv  Detail & Related papers  (2024-08-12T11:47:11Z)
• I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models [18.36472998650704]
  We introduce a novel and generic solution that extends the applicability of image editing tools to videos by propagating edits from a single frame to the entire video using a pre-trained image-to-video model. Our method, dubbed I2VEdit, adaptively preserves the visual and motion integrity of the source video depending on the extent of the edits.
  arXiv  Detail & Related papers  (2024-05-26T11:47:40Z)
• GenDeF: Learning Generative Deformation Field for Video Generation [89.49567113452396]
  We propose to render a video by warping one static image with a generative deformation field (GenDeF) Such a pipeline enjoys three appealing advantages.
  arXiv  Detail & Related papers  (2023-12-07T18:59:41Z)
• DynVideo-E: Harnessing Dynamic NeRF for Large-Scale Motion- and View-Change Human-Centric Video Editing [48.086102360155856]
  We introduce the dynamic Neural Radiance Fields (NeRF) as the innovative video representation. We propose the image-based video-NeRF editing pipeline with a set of innovative designs to provide consistent and controllable editing. Our method, dubbed as DynVideo-E, significantly outperforms SOTA approaches on two challenging datasets by a large margin of 50% 95% for human preference.
  arXiv  Detail & Related papers  (2023-10-16T17:48:10Z)
• Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models [65.268245109828]
  Ground-A-Video is a video-to-video translation framework for multi-attribute video editing. It attains temporally consistent editing of input videos in a training-free manner. Experiments and applications demonstrate that Ground-A-Video's zero-shot capacity outperforms other baseline methods in terms of edit-accuracy and frame consistency.
  arXiv  Detail & Related papers  (2023-10-02T11:28:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.