Explicit Layer Modeling for Video Object Insertion and Layer Decomposition

• URL: http://arxiv.org/abs/2607.25802v2
• Date: Wed, 29 Jul 2026 07:28:17 GMT
• Title: Explicit Layer Modeling for Video Object Insertion and Layer Decomposition
• Abstract summary: We introduce TriLayer, where the foreground layers include both object appearance and associated visual effects.<n>This explicit supervision enables models to learn layered video representations directly rather than inferring them implicitly.<n>We propose DBL-Diffusion, a dual-branch diffusion framework that jointly models RGB composites and RGBA foreground layers.
• Score: 18.022394461124
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Most video editing systems still lack explicit layered video representations, limiting their ability to perform realistic compositing, object reuse, and consistent manipulation. This limitation is especially pronounced in video object insertion and video layer decomposition, where existing methods rely on implicit inference or per-scene optimization due to the absence of explicit foreground-layer supervision. We introduce TriLayer, a large-scale triplet video dataset containing aligned composite, background, and foreground videos, where the foreground layers include both object appearance and associated visual effects. This explicit supervision enables models to learn layered video representations directly rather than inferring them implicitly. Building on this dataset, we propose DBL-Diffusion, a dual-branch diffusion framework that jointly models RGB composites and RGBA foreground layers through shared denoising and cross-branch interaction. We instantiate the framework in two tasks: DBL-Insert for layered object insertion, which generates explicit RGBA layers for realistic compositing and flexible post-editing, and DBL-Decompose for video layer decomposition, which recovers foreground and background layers using triplet supervision. Experiments demonstrate that explicit layer modeling substantially improves both insertion fidelity and decomposition quality.

Related papers

• Referring Layer Decomposition [25.128453386102887]
  We introduce the Referring Layer Decomposition (RLD) task, which predicts complete RGBA layers from a single RGB image.<n>At the core is the RefLade, a large-scale dataset comprising 1.11M image-layer-prompt triplets produced by our scalable data engine.<n>We present RefLayer, a simple baseline designed for prompt-conditioned layer decomposition, achieving high visual fidelity and semantic alignment.
  arXiv  Detail & Related papers  (2026-02-22T22:05:17Z)
• Qwen-Image-Layered: Towards Inherent Editability via Layer Decomposition [73.43121650616804]
  We propose textbfQwen-Image-Layered, an end-to-end diffusion model that decomposes a single RGB image into multiple semantically disentangled RGBA layers.<n>Our method significantly surpasses existing approaches in decomposition quality and establishes a new paradigm for consistent image editing.
  arXiv  Detail & Related papers  (2025-12-17T17:12:42Z)
• From Inpainting to Layer Decomposition: Repurposing Generative Inpainting Models for Image Layer Decomposition [16.7393689710179]
  layered representation enables independent editing of elements, offering greater flexibility for content creation.<n>We observe a strong connection between layer decomposition and in/outpainting tasks, and propose adapting a diffusion-based inpainting model for layer decomposition using lightweight finetuning.<n>To further preserve detail in the latent space, we introduce a novel multi-modal context fusion module with linear attention complexity.
  arXiv  Detail & Related papers  (2025-11-26T02:50:07Z)
• BlenderFusion: 3D-Grounded Visual Editing and Generative Compositing [39.18857645517109]
  We present BlenderFusion, a generative visual compositing framework that synthesizes new scenes by recomposing objects, camera, and background.<n>It follows a layering-editing-compositing pipeline: (i) segmenting and converting visual inputs into editable 3D entities (layering), (ii) editing them in Blender with 3D-grounded control (editing), and (iii) fusing them into a coherent scene using a generative compositor (compositing)
  arXiv  Detail & Related papers  (2025-06-20T19:38:34Z)
• DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers [85.1185656296496]
  We present DiffDecompose, a diffusion Transformer-based framework that learns the posterior over possible layer decompositions conditioned on the input image.<n>The code and dataset will be available upon paper acceptance.
  arXiv  Detail & Related papers  (2025-05-24T16:08:04Z)
• BlobGEN-Vid: Compositional Text-to-Video Generation with Blob Video Representations [82.94002870060045]
  Existing video generation models struggle to follow complex text prompts and synthesize multiple objects.<n>We develop a blob-grounded video diffusion model named BlobGEN-Vid that allows users to control object motions and fine-grained object appearance.<n>We show that our framework is model-agnostic and build BlobGEN-Vid based on both U-Net and DiT-based video diffusion models.
  arXiv  Detail & Related papers  (2025-01-13T19:17:06Z)
• Video Decomposition Prior: A Methodology to Decompose Videos into Layers [74.36790196133505]
  This paper introduces a novel video decomposition prior VDP' framework which derives inspiration from professional video editing practices.<n>VDP framework decomposes a video sequence into a set of multiple RGB layers and associated opacity levels.<n>We address tasks such as video object segmentation, dehazing, and relighting.
  arXiv  Detail & Related papers  (2024-12-06T10:35:45Z)
• InVi: Object Insertion In Videos Using Off-the-Shelf Diffusion Models [46.587906540660455]
  We introduce InVi, an approach for inserting or replacing objects within videos using off-the-shelf, text-to-image latent diffusion models. InVi achieves realistic object insertion with consistent blending and coherence across frames, outperforming existing methods.
  arXiv  Detail & Related papers  (2024-07-15T17:55:09Z)
• WALDO: Future Video Synthesis using Object Layer Decomposition and Parametric Flow Prediction [82.79642869586587]
  WALDO is a novel approach to the prediction of future video frames from past ones. Individual images are decomposed into multiple layers combining object masks and a small set of control points. The layer structure is shared across all frames in each video to build dense inter-frame connections.
  arXiv  Detail & Related papers  (2022-11-25T18:59:46Z)
• Unsupervised Video Interpolation by Learning Multilayered 2.5D Motion Fields [75.81417944207806]
  This paper presents a self-supervised approach to video frame learning that requires only a single video. We parameterize the video motions by solving an ordinary differentiable equation (ODE) defined on a time-varying motion field. This implicit neural representation learns the video as a space-time continuum, allowing frame-time continuum at any temporal resolution.
  arXiv  Detail & Related papers  (2022-04-21T06:17:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.