Vera: A Layered Diffusion Model for Content-Preserving Video Editing

• URL: http://arxiv.org/abs/2606.23610v1
• Date: Mon, 22 Jun 2026 17:11:11 GMT
• Title: Vera: A Layered Diffusion Model for Content-Preserving Video Editing
• Abstract summary: We introduce Vera, a layered diffusion framework for content-preserving video editing.<n>Instead of regenerating the entire video, Vera generates an edit layer along with an alpha matte for compositing with the source video.<n>Vera outperforms leading open-source video editing models in content preservation while remaining competitive in edit quality, using 486K frames of layered training data.
• Score: 31.27751797441043
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Video diffusion models have enabled remarkable progress in video generation and editing. However, content preservation remains a core challenge: existing methods regenerate every pixel and often alter elements that should remain unchanged, such as characters or background scenes. We introduce Vera, a layered diffusion framework for content-preserving video editing. Instead of regenerating the entire video, Vera generates an edit layer along with an alpha matte for compositing with the source video, separating creative editing from content preservation by design. To encourage coherent composition with the source video, we extend the text-to-video DiT into a Mixture-of-Transformers (MoT) architecture, with separate DiTs for each layer that interact through joint self-attention. To support the training of Vera, we further construct a high-quality layered dataset with accurate alpha mattes, diverse scenes and dynamics, and visual effects. Across our quantitative benchmark and human preference study, Vera outperforms leading open-source video editing models in content preservation while remaining competitive in edit quality, using 486K frames of layered training data.

Related papers

• Instruction-Based Video Editing by Repurposing an Image Editing Model [55.9214783600876]
  We show that a strong instruction-based image editing model can edit videos by operating directly on video-VAE latents.<n>Our results suggest that, despite the large investment in training video latent spaces, per-frame video latents remain close enough to the image domain that mature image editing priors transfer with minimal adaptation.
  arXiv  Detail & Related papers  (2026-08-14T18:01:29Z)
• DiffMagicFace: Identity Consistent Facial Editing of Real Videos [16.262125513013746]
  We introduce DiffMagicFace, a unique video editing framework that integrates two fine-tuned models for text and image control.<n>Our approach does not depend on video datasets but still delivers high-quality results in both consistency and content.
  arXiv  Detail & Related papers  (2026-04-15T13:13:04Z)
• Layer-Aware Video Composition via Split-then-Merge [55.12521724893102]
  Split-then-Merge (StM) is a framework designed to enhance control in generative video composition.<n>StM splits a large corpus of unlabeled videos into dynamic foreground and background layers, then self-composes them to learn how dynamic subjects interact with diverse scenes.
  arXiv  Detail & Related papers  (2025-11-25T19:53:15Z)
• EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning [58.53074381801114]
  We introduce EditVerse, a unified framework for image and video generation and editing within a single model.<n>By representing all modalities, i.e. text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning.<n>We present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions.
  arXiv  Detail & Related papers  (2025-09-24T17:59:30Z)
• GenCompositor: Generative Video Compositing with Diffusion Transformer [68.00271033575736]
  Traditional pipelines require intensive labor efforts and expert collaboration, resulting in lengthy production cycles and high manpower costs.<n>This new task strives to adaptively inject identity and motion information of foreground video to the target video in an interactive manner.<n>Experiments demonstrate that our method effectively realizes generative video compositing, outperforming existing possible solutions in fidelity and consistency.
  arXiv  Detail & Related papers  (2025-09-02T16:10:13Z)
• TV-LiVE: Training-Free, Text-Guided Video Editing via Layer Informed Vitality Exploitation [36.81368812919819]
  We present TV-LiVE, a Training-free and text-guided Video editing framework via Layerinformed Vitality Exploitation.<n>We empirically identify vital layers within the video generation model that significantly influence the quality of generated outputs.<n>For object addition, we identify prominent layers to extract the mask regions corresponding to the newly added target prompt.
  arXiv  Detail & Related papers  (2025-06-08T16:12:13Z)
• UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editing [28.140945021777878]
  We present UniEdit, a tuning-free framework that supports both video motion and appearance editing. To realize motion editing while preserving source video content, we introduce auxiliary motion-reference and reconstruction branches. The obtained features are then injected into the main editing path via temporal and spatial self-attention layers.
  arXiv  Detail & Related papers  (2024-02-20T17:52:12Z)
• Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models [65.268245109828]
  Ground-A-Video is a video-to-video translation framework for multi-attribute video editing. It attains temporally consistent editing of input videos in a training-free manner. Experiments and applications demonstrate that Ground-A-Video's zero-shot capacity outperforms other baseline methods in terms of edit-accuracy and frame consistency.
  arXiv  Detail & Related papers  (2023-10-02T11:28:37Z)
• InFusion: Inject and Attention Fusion for Multi Concept Zero-Shot Text-based Video Editing [27.661609140918916]
  InFusion is a framework for zero-shot text-based video editing. It supports editing of multiple concepts with pixel-level control over diverse concepts mentioned in the editing prompt. Our framework is a low-cost alternative to one-shot tuned models for editing since it does not require training.
  arXiv  Detail & Related papers  (2023-07-22T17:05:47Z)
• VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing [18.24307442582304]
  We introduce VidEdit, a novel method for zero-shot text-based video editing. Our experiments show that VidEdit outperforms state-of-the-art methods on DAVIS dataset.
  arXiv  Detail & Related papers  (2023-06-14T19:15:49Z)
• Edit-A-Video: Single Video Editing with Object-Aware Consistency [49.43316939996227]
  We propose a video editing framework given only a pretrained TTI model and a single text, video> pair, which we term Edit-A-Video. The framework consists of two stages: (1) inflating the 2D model into the 3D model by appending temporal modules tuning and on the source video (2) inverting the source video into the noise and editing with target text prompt and attention map injection. We present extensive experimental results over various types of text and videos, and demonstrate the superiority of the proposed method compared to baselines in terms of background consistency, text alignment, and video editing quality.
  arXiv  Detail & Related papers  (2023-03-14T14:35:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.