Tuning-free Instruction-based Video Editing Via Structural Noise Initialization and Guidance

• URL: http://arxiv.org/abs/2605.15533v1
• Date: Fri, 15 May 2026 02:09:06 GMT
• Title: Tuning-free Instruction-based Video Editing Via Structural Noise Initialization and Guidance
• Abstract summary: We propose a textittuning-free, instruction-based video editing framework.<n>Our proposed method achieves better visual quality and state-of-the-art performance.
• Score: 34.63499907054888
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Video editing poses a significant challenge. While a series of tuning-free methods circumvent the need for extensive data collection and model training, they often underutilize the rich information embedded within noisy latent, leading to unsatisfactory results. To address this, we propose a \textit{tuning-free, instruction-based} video editing framework. We approach video editing from the perspective of noisy latent: we design a Structural Noise Initialization Strategy (SNIS) to secure a superior editing starting point by assigning higher noise levels to edited regions (to facilitate content change) and lower noise levels to unedited regions (to maintain content consistency). We introduce a Noise Guidance Mechanism (NGM), which leverages the video prior in the generative model and effectively integrates rich information within the noisy latent to guide the denoising process, thereby preserving unedited content and overall visual coherence. Experiments show that our proposed method achieves better visual quality and state-of-the-art performance.

Related papers

• Region-Constraint In-Context Generation for Instructional Video Editing [91.27224696009755]
  We present ReCo, a new instructional video editing paradigm that delves into constraint modeling between editing and non-editing regions during in-context generation.<n>We propose a large-scale, high-quality video editing dataset, i.e., ReCo-Data, comprising 500K instruction-video pairs to benefit model training.
  arXiv  Detail & Related papers  (2025-12-19T14:49:30Z)
• Coherent Audio-Visual Editing via Conditional Audio Generation Following Video Edits [33.1393328136321]
  We introduce a novel pipeline for joint audio-visual editing that enhances the coherence between edited video and its accompanying audio.<n>Our approach first applies state-of-the-art video editing techniques to produce the target video, then performs audio editing to align with the visual changes.
  arXiv  Detail & Related papers  (2025-12-08T06:45:11Z)
• Object-AVEdit: An Object-level Audio-Visual Editing Model [79.62095842136115]
  We present textbfObject-AVEdit, achieving the object-level audio-visual editing based on the inversion-regeneration paradigm.<n>To achieve the object-level controllability during editing, we develop a word-to-sounding-object well-aligned audio generation model.<n>To achieve the better structural information preservation and object-level editing effect, we propose an inversion-regeneration holistically-optimized editing algorithm.
  arXiv  Detail & Related papers  (2025-09-27T18:12:13Z)
• Taming Flow-based I2V Models for Creative Video Editing [64.67801702413122]
  Video editing, which aims to manipulate videos according to user intent, remains an emerging challenge.<n>Most existing image-conditioned video editing methods require inversion with model-specific design or need extensive optimization.<n>We propose IF-V2V, an Inversion-Free method that can adapt off-the-shelf flow-matching-based I2V models for video editing without significant computational overhead.
  arXiv  Detail & Related papers  (2025-09-26T05:57:04Z)
• DNI: Dilutional Noise Initialization for Diffusion Video Editing [22.308638156328968]
  This paper proposes Dilutional Noise Initialization (DNI) framework which enables editing systems to perform precise and dynamic modification including non-rigid editing. DNI introduces a concept of noise dilution' which adds further noise to the latent noise in the region to be edited to soften the structural rigidity imposed by input video.
  arXiv  Detail & Related papers  (2024-09-19T18:17:57Z)
• TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models [53.757752110493215]
  We focus on a popular line of text-based editing frameworks - the edit-friendly'' DDPM-noise inversion approach. We analyze its application to fast sampling methods and categorize its failures into two classes: the appearance of visual artifacts, and insufficient editing strength. We propose a pseudo-guidance approach that efficiently increases the magnitude of edits without introducing new artifacts.
  arXiv  Detail & Related papers  (2024-08-01T17:27:28Z)
• Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models [47.518487213173785]
  We propose a novel formulation that considers both visual quality and consistency of content. Consistency of content is ensured by a proposed loss function that maintains the structure of the input, while visual quality is improved by utilizing the denoising process of pretrained diffusion models.
  arXiv  Detail & Related papers  (2024-07-14T17:59:56Z)
• Zero-Shot Video Editing through Adaptive Sliding Score Distillation [51.57440923362033]
  This study proposes a novel paradigm of video-based score distillation, facilitating direct manipulation of original video content. We propose an Adaptive Sliding Score Distillation strategy, which incorporates both global and local video guidance to reduce the impact of editing errors.
  arXiv  Detail & Related papers  (2024-06-07T12:33:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.