DFVEdit: Conditional Delta Flow Vector for Zero-shot Video Editing

• URL: http://arxiv.org/abs/2506.20967v2
• Date: Fri, 27 Jun 2025 08:42:17 GMT
• Title: DFVEdit: Conditional Delta Flow Vector for Zero-shot Video Editing
• Authors: Lingling Cai, Kang Zhao, Hangjie Yuan, Xiang Wang, Yingya Zhang, Kejie Huang,
• Abstract summary: We present DFVEdit, an efficient zero-shot video editing method tailored for Video DiTs.<n>DFVEdit eliminates the need for both attention modification and fine-tuning by directly operating on clean latents via flow transformation.<n>Building upon this foundation, we propose the Conditional Delta Flow Vector (CDFV) and integrate Implicit Cross Attention (ICA) guidance.
• Score: 18.86599058385878
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The advent of Video Diffusion Transformers (Video DiTs) marks a milestone in video generation. However, directly applying existing video editing methods to Video DiTs often incurs substantial computational overhead, due to resource-intensive attention modification or finetuning. To alleviate this problem, we present DFVEdit, an efficient zero-shot video editing method tailored for Video DiTs. DFVEdit eliminates the need for both attention modification and fine-tuning by directly operating on clean latents via flow transformation. To be more specific, we observe that editing and sampling can be unified under the continuous flow perspective. Building upon this foundation, we propose the Conditional Delta Flow Vector (CDFV) -- a theoretically unbiased estimation of DFV -- and integrate Implicit Cross Attention (ICA) guidance as well as Embedding Reinforcement (ER) to further enhance editing quality. DFVEdit excels in practical efficiency, offering at least 20x inference speed-up and 85% memory reduction on Video DiTs compared to attention-engineering-based editing methods. Extensive quantitative and qualitative experiments demonstrate that DFVEdit can be seamlessly applied to popular Video DiTs (e.g., CogVideoX and Wan2.1), attaining state-of-the-art performance on structural fidelity, spatial-temporal consistency, and editing quality.

Related papers

• Consistent Video Editing as Flow-Driven Image-to-Video Generation [6.03121849763522]
  FlowV2V decomposes the entire pipeline into first-frame editing and conditional I2V generation, and simulates pseudo flow sequence that aligns with the deformed shape.<n> Experimental results on DAVIS-EDIT with improvements of 13.67% and 50.66% on DOVER and warping error illustrate the superior temporal consistency and sample quality of FlowV2V compared to existing state-of-the-art ones.
  arXiv  Detail & Related papers  (2025-06-09T12:57:30Z)
• FiVE: A Fine-grained Video Editing Benchmark for Evaluating Emerging Diffusion and Rectified Flow Models [17.788970036356297]
  We introduce FiVE, a Fine-grained Video Editing Benchmark for evaluating emerging diffusion and rectified flow models.<n>Our benchmark includes 74 real-world videos and 26 generated videos, featuring 6 fine-grained editing types, 420 object-level editing prompt pairs, and their corresponding masks.<n>We evaluate five diffusion-based and two RF-based editing methods on our FiVE benchmark using 15 metrics, covering background preservation, text-video similarity, temporal consistency, video quality, and runtime.
  arXiv  Detail & Related papers  (2025-03-17T19:47:41Z)
• Taming Rectified Flow for Inversion and Editing [57.3742655030493]
  Rectified-flow-based diffusion transformers like FLUX and OpenSora have demonstrated outstanding performance in the field of image and video generation.<n>Despite their robust generative capabilities, these models often struggle with inaccuracies.<n>We propose RF-r, a training-free sampler that effectively enhances inversion precision by mitigating the errors in the inversion process of rectified flow.
  arXiv  Detail & Related papers  (2024-11-07T14:29:02Z)
• COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editing [57.76170824395532]
  Video editing is an emerging task, in which most current methods adopt the pre-trained text-to-image (T2I) diffusion model to edit the source video.<n>We propose COrrespondence-guided Video Editing (COVE) to achieve high-quality and consistent video editing.<n>COVE can be seamlessly integrated into the pre-trained T2I diffusion model without the need for extra training or optimization.
  arXiv  Detail & Related papers  (2024-06-13T06:27:13Z)
• Zero-Shot Video Editing through Adaptive Sliding Score Distillation [51.57440923362033]
  This study proposes a novel paradigm of video-based score distillation, facilitating direct manipulation of original video content. We propose an Adaptive Sliding Score Distillation strategy, which incorporates both global and local video guidance to reduce the impact of editing errors.
  arXiv  Detail & Related papers  (2024-06-07T12:33:59Z)
• EffiVED:Efficient Video Editing via Text-instruction Diffusion Models [9.287394166165424]
  We introduce EffiVED, an efficient diffusion-based model that supports instruction-guided video editing. We transform vast image editing datasets and open-world videos into a high-quality dataset for training EffiVED.
  arXiv  Detail & Related papers  (2024-03-18T08:42:08Z)
• FastVideoEdit: Leveraging Consistency Models for Efficient Text-to-Video Editing [8.907836546058086]
  Existing approaches relying on image generation models for video editing suffer from time-consuming one-shot fine-tuning, additional condition extraction, or DDIM inversion. We propose FastVideoEdit, an efficient zero-shot video editing approach inspired by Consistency Models (CMs) Our method enables direct mapping from source video to target video with strong preservation ability utilizing a special variance schedule.
  arXiv  Detail & Related papers  (2024-03-10T17:12:01Z)
• Fuse Your Latents: Video Editing with Multi-source Latent Diffusion Models [66.12367865049572]
  Latent Diffusion Models (LDMs) are renowned for their powerful capabilities in image and video synthesis. We propose FLDM, a framework that achieves high-quality text-to-video (T2V) editing by integrating various T2I and T2V LDMs. This paper is the first to reveal that T2I and T2V LDMs can complement each other in terms of structure and temporal consistency.
  arXiv  Detail & Related papers  (2023-10-25T06:35:01Z)
• Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models [65.268245109828]
  Ground-A-Video is a video-to-video translation framework for multi-attribute video editing. It attains temporally consistent editing of input videos in a training-free manner. Experiments and applications demonstrate that Ground-A-Video's zero-shot capacity outperforms other baseline methods in terms of edit-accuracy and frame consistency.
  arXiv  Detail & Related papers  (2023-10-02T11:28:37Z)
• Edit Temporal-Consistent Videos with Image Diffusion Model [49.88186997567138]
  Large-scale text-to-image (T2I) diffusion models have been extended for text-guided video editing. T achieves state-of-the-art performance in both video temporal consistency and video editing capability.
  arXiv  Detail & Related papers  (2023-08-17T16:40:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.