Controllable Video Object Insertion via Multiview Priors

• URL: http://arxiv.org/abs/2604.14556v1
• Date: Thu, 16 Apr 2026 02:39:15 GMT
• Title: Controllable Video Object Insertion via Multiview Priors
• Abstract summary: Video object insertion is a critical task for dynamically inserting new objects into existing environments.<n>We propose a novel solution for Video Object Insertion, which integrates multi-view object priors.<n>We show that our solution significantly improves the quality of video object insertion, providing stable and realistic integration.
• Score: 30.363004603304418
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Video object insertion is a critical task for dynamically inserting new objects into existing environments. Previous video generation methods focus primarily on synthesizing entire scenes while struggling with ensuring consistent object appearance, spatial alignment, and temporal coherence when inserting objects into existing videos. In this paper, we propose a novel solution for Video Object Insertion, which integrates multi-view object priors to address the common challenges of appearance inconsistency and occlusion handling in dynamic environments. By lifting 2D reference images into multi-view representations and leveraging a dual-path view-consistent conditioning mechanism, our framework ensures stable identity guidance and robust integration across diverse viewpoints. A quality-aware weighting mechanism is also employed to adaptively handle noisy or imperfect inputs. Additionally, we introduce an Integration-Aware Consistency Module that guarantees spatial realism, effectively resolving occlusion and boundary artifacts while maintaining temporal continuity across frames. Experimental results show that our solution significantly improves the quality of video object insertion, providing stable and realistic integration.

Related papers

• FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control [26.208912960696622]
  We propose a unified framework that standardizes video compositing as a trajectory-guided conditional generation task.<n>Our approach introduces three key designs: (1) a Unified Foreground Representation that decouples an object's intrinsic motion from its global displacement, standardizing heterogeneous inputs into a stabilized, centered latent space; (2) a Spatial-Aware Latent Injection strategy that exploits the translation equivariance of VAE latent spaces to transport canonical features onto target trajectories via a parameter-free mechanism; and (3) a Hybrid dataset and Synthetic-to-Real Curriculum that synergizes procedural simulation, real-world cinematic footage, and generative data to
  arXiv  Detail & Related papers  (2026-07-31T16:59:46Z)
• Rethinking Object-Centric Representations for Video Dynamics Modeling [17.157587526107598]
  Unsupervised video object tracking aims to decompose dynamic scenes into persistent, object-centric entities without manual annotations.<n>Many recent approaches rely on slot-based representations, where a fixed set of latent variables ("slots") represent individual objects across frames.<n>We propose STAITUS, a unified framework that explicitly disentangles each slot into appearance and geometric pose (position/scale).
  arXiv  Detail & Related papers  (2026-06-22T14:55:41Z)
• Smart-Insertion-V: Photorealistic Video Insertion via a Closed-Loop Feedback Dual-Stream Framework [52.477407591581176]
  Mask-free video object insertion has emerged as a challenging task, requiring harmonious integration of reference objects into source videos.<n>We propose textittextbfSmart-Insertion-V, an end-to-end textbfDual-Stream framework that concurrently conducts video insertion and image style transfer.
  arXiv  Detail & Related papers  (2026-05-22T17:54:54Z)
• AGILE: Hand-Object Interaction Reconstruction from Video via Agentic Generation [45.753757870577196]
  We introduce AGILE, a robust framework that shifts the paradigm from reconstruction to agentic generation for interaction learning.<n>We show that AGILE outperforms baselines in global geometric accuracy while demonstrating exceptional robustness on challenging sequences where prior art frequently collapses.
  arXiv  Detail & Related papers  (2026-02-04T15:42:58Z)
• PLACID: Identity-Preserving Multi-Object Compositing via Video Diffusion with Synthetic Trajectories [22.63777279327245]
  PLACID is a framework that transforms a collection of object images into an appealing multi-object composite.<n>First, we leverage a pretrained image-to-video (I2V) diffusion model with text control to preserve objects consistency, identities, and background details.<n>Second, we propose a novel data curation strategy that generates synthetic sequences where randomly placed objects smoothly move to their target positions.
  arXiv  Detail & Related papers  (2026-01-30T19:42:54Z)
• Learning Object-Centric Representations Based on Slots in Real World Scenarios [5.922488908114023]
  This thesis introduces a framework that adapts powerful pretrained diffusion models for object-centric synthesis.<n>We identify a core challenge: balancing global scene coherence with disentangled object control.<n>Our method integrates lightweight, slot-based conditioning into pretrained models, preserving their visual priors while providing object-specific manipulation.
  arXiv  Detail & Related papers  (2025-09-29T12:01:49Z)
• Compositional Video Synthesis by Temporal Object-Centric Learning [3.2228025627337864]
  We present a novel framework for compositional video synthesis that leverages temporally consistent object-centric representations.<n>Our approach explicitly captures temporal dynamics by learning pose invariant object-centric slots and conditioning them on pretrained diffusion models.<n>This design enables high-quality, pixel-level video synthesis with superior temporal coherence.
  arXiv  Detail & Related papers  (2025-07-28T14:11:04Z)
• Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Control [72.00655365269]
  We present RoboMaster, a novel framework that models inter-object dynamics through a collaborative trajectory formulation.<n>Unlike prior methods that decompose objects, our core is to decompose the interaction process into three sub-stages: pre-interaction, interaction, and post-interaction.<n>Our method outperforms existing approaches, establishing new state-of-the-art performance in trajectory-controlled video generation for robotic manipulation.
  arXiv  Detail & Related papers  (2025-06-02T17:57:06Z)
• HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation [50.206100327643284]
  HiScene is a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation.<n>We generate 3D content that aligns with 2D representations while maintaining compositional structure.
  arXiv  Detail & Related papers  (2025-04-17T16:33:39Z)
• Through-The-Mask: Mask-based Motion Trajectories for Image-to-Video Generation [52.337472185022136]
  We consider the task of Image-to-Video (I2V) generation, which involves transforming static images into realistic video sequences based on a textual description.<n>We propose a two-stage compositional framework that decomposes I2V generation into: (i) An explicit intermediate representation generation stage, followed by (ii) A video generation stage that is conditioned on this representation.<n>We evaluate our method on challenging benchmarks with multi-object and high-motion scenarios and empirically demonstrate that the proposed method achieves state-of-the-art consistency.
  arXiv  Detail & Related papers  (2025-01-06T14:49:26Z)
• VideoSAM: Open-World Video Segmentation [33.09103541421983]
  We introduce VideoSAM, an end-to-end framework designed to improve object tracking and segmentation consistency in dynamic environments. VideoSAM integrates an agglomerated backbone, RADIO, enabling object association through similarity metrics. Our method is extensively evaluated on the UVO and BURST benchmarks, and robotic videos from RoboTAP, demonstrating its effectiveness and robustness in real-world scenarios.
  arXiv  Detail & Related papers  (2024-10-11T12:56:32Z)
• Training-Free Robust Interactive Video Object Segmentation [82.05906654403684]
  We propose a training-free prompt tracking framework for interactive video object segmentation (I-PT) We jointly adopt sparse points and boxes tracking, filtering out unstable points and capturing object-wise information. Our framework has demonstrated robust zero-shot video segmentation results on popular VOS datasets.
  arXiv  Detail & Related papers  (2024-06-08T14:25:57Z)
• Object-Centric Image Generation from Layouts [93.10217725729468]
  We develop a layout-to-image-generation method to generate complex scenes with multiple objects. Our method learns representations of the spatial relationships between objects in the scene, which lead to our model's improved layout-fidelity. We introduce SceneFID, an object-centric adaptation of the popular Fr'echet Inception Distance metric, that is better suited for multi-object images.
  arXiv  Detail & Related papers  (2020-03-16T21:40:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.