Seen-to-Scene: Keep the Seen, Generate the Unseen for Video Outpainting

• URL: http://arxiv.org/abs/2604.14648v1
• Date: Thu, 16 Apr 2026 05:55:11 GMT
• Title: Seen-to-Scene: Keep the Seen, Generate the Unseen for Video Outpainting
• Abstract summary: Video outpainting aims to expand the visible content of a video beyond the original frame boundaries.<n>We propose Seen-to-Scene, a novel framework that unifies propagationbased and generation-based paradigms for video outpainting.
• Score: 29.60843145809862
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video outpainting aims to expand the visible content of a video beyond the original frame boundaries while preserving spatial fidelity and temporal coherence across frames. Existing methods primarily rely on large-scale generative models, such as diffusion models. However, generationbased approaches suffer from implicit temporal modeling and limited spatial context. These limitations lead to intraframe and inter-frame inconsistencies, which become particularly pronounced in dynamic scenes and large outpainting scenarios. To overcome these challenges, we propose Seen-to-Scene, a novel framework that unifies propagationbased and generation-based paradigms for video outpainting. Specifically, Seen-to-Scene leverages flow-based propagation with a flow completion network pre-trained for video inpainting, which is fine-tuned in an end-to-end manner to bridge the domain gap and reconstruct coherent motion fields. To further improve the efficiency and reliability of propagation, we introduce a reference-guided latent propagation that effectively propagates source content across frames. Extensive experiments demonstrate that our method achieves superior temporal coherence and visual realism with efficient inference, surpassing even prior state-of-the-art methods that require input-specific adaptation.

Related papers

• Zero-Shot Video Translation and Editing with Frame Spatial-Temporal Correspondence [81.82643953694485]
  We present FRESCO, which integrates intra-frame correspondence with inter-frame correspondence to formulate a more robust spatial-temporal constraint.<n>Our method goes beyond attention guidance to explicitly optimize features, achieving high spatial-temporal consistency with the input video.<n>We verify FRESCO adaptations on two zero-shot tasks of video-to-video translation and text-guided video editing.
  arXiv  Detail & Related papers  (2025-12-03T15:51:11Z)
• VidSplice: Towards Coherent Video Inpainting via Explicit Spaced Frame Guidance [57.57195766748601]
  VidSplice is a novel framework that guides inpainting process withtemporal cues.<n>We show that VidSplice achieves competitive performance across diverse video inpainting scenarios.
  arXiv  Detail & Related papers  (2025-10-24T13:44:09Z)
• DiffuEraser: A Diffusion Model for Video Inpainting [13.292164408616257]
  We introduce DiffuEraser, a video inpainting model based on stable diffusion, to fill masked regions with greater details and more coherent structures.<n>We also expand the temporal receptive fields of both the prior model and DiffuEraser, and further enhance consistency by leveraging the temporal smoothing property of Video Diffusion Models.
  arXiv  Detail & Related papers  (2025-01-17T08:03:02Z)
• RepVideo: Rethinking Cross-Layer Representation for Video Generation [53.701548524818534]
  We propose RepVideo, an enhanced representation framework for text-to-video diffusion models.<n>By accumulating features from neighboring layers to form enriched representations, this approach captures more stable semantic information.<n>Our experiments demonstrate that our RepVideo not only significantly enhances the ability to generate accurate spatial appearances, but also improves temporal consistency in video generation.
  arXiv  Detail & Related papers  (2025-01-15T18:20:37Z)
• Optical-Flow Guided Prompt Optimization for Coherent Video Generation [51.430833518070145]
  We propose a framework called MotionPrompt that guides the video generation process via optical flow.<n>We optimize learnable token embeddings during reverse sampling steps by using gradients from a trained discriminator applied to random frame pairs.<n>This approach allows our method to generate visually coherent video sequences that closely reflect natural motion dynamics, without compromising the fidelity of the generated content.
  arXiv  Detail & Related papers  (2024-11-23T12:26:52Z)
• Semantically Consistent Video Inpainting with Conditional Diffusion Models [16.42354856518832]
  We present a framework for solving problems with conditional video diffusion models. We introduce inpainting-specific sampling schemes which capture crucial long-range dependencies in the context. We devise a novel method for conditioning on the known pixels in incomplete frames.
  arXiv  Detail & Related papers  (2024-04-30T23:49:26Z)
• FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation [85.29772293776395]
  We introduce FRESCO, intra-frame correspondence alongside inter-frame correspondence to establish a more robust spatial-temporal constraint. This enhancement ensures a more consistent transformation of semantically similar content across frames. Our approach involves an explicit update of features to achieve high spatial-temporal consistency with the input video.
  arXiv  Detail & Related papers  (2024-03-19T17:59:18Z)
• Training-Free Semantic Video Composition via Pre-trained Diffusion Model [96.0168609879295]
  Current approaches, predominantly trained on videos with adjusted foreground color and lighting, struggle to address deep semantic disparities beyond superficial adjustments. We propose a training-free pipeline employing a pre-trained diffusion model imbued with semantic prior knowledge. Experimental results reveal that our pipeline successfully ensures the visual harmony and inter-frame coherence of the outputs.
  arXiv  Detail & Related papers  (2024-01-17T13:07:22Z)
• Internal Video Inpainting by Implicit Long-range Propagation [39.89676105875726]
  We propose a novel framework for video inpainting by adopting an internal learning strategy. We show that this can be achieved implicitly by fitting a convolutional neural network to the known region. We extend the proposed method to another challenging task: learning to remove an object from a video giving a single object mask in only one frame in a 4K video.
  arXiv  Detail & Related papers  (2021-08-04T08:56:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.