Voyaging into Unbounded Dynamic Scenes from a Single View

• URL: http://arxiv.org/abs/2507.04183v1
• Date: Sat, 05 Jul 2025 22:49:25 GMT
• Title: Voyaging into Unbounded Dynamic Scenes from a Single View
• Authors: Fengrui Tian, Tianjiao Ding, Jinqi Luo, Hancheng Min, René Vidal,
• Abstract summary: We propose DynamicVoyager that reformulates the dynamic scene generation as a scene outpainting process for new dynamic content.<n>We render the partial video at a novel view and outpaint the video with ray contexts from the point cloud to generate 3D consistent motions.<n>Experiments show that our model is able to generate unbounded scenes with consistent motions along fly-through cameras.
• Score: 31.85867311855001
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper studies the problem of generating an unbounded dynamic scene from a single view, which has wide applications in augmented/virtual reality and robotics. Since the scene is changing over time, different generated views need to be consistent with the underlying 3D motions. While previous works learn such consistency by training from multiple views, the generated scene regions are bounded to be close to the training views with limited camera movements. To address this issue, we propose DynamicVoyager that reformulates the dynamic scene generation as a scene outpainting process for new dynamic content. As 2D outpainting models can hardly generate 3D consistent motions from only 2D pixels at a single view, we consider pixels as rays to enrich the pixel input with the ray context, so that the 3D motion consistency can be learned from the ray information. More specifically, we first map the single-view video input to a dynamic point cloud with the estimated video depths. Then we render the partial video at a novel view and outpaint the video with ray contexts from the point cloud to generate 3D consistent motions. We employ the outpainted video to update the point cloud, which is used for scene outpainting from future novel views. Experiments show that our model is able to generate unbounded scenes with consistent motions along fly-through cameras, and the generated contents can be controlled with scene prompts.

Related papers

• DreamJourney: Perpetual View Generation with Video Diffusion Models [91.88716097573206]
  Perpetual view generation aims to synthesize a long-term video corresponding to an arbitrary camera trajectory solely from a single input image.<n>Recent methods commonly utilize a pre-trained text-to-image diffusion model to synthesize new content of previously unseen regions along camera movement.<n>We present DreamJourney, a two-stage framework that leverages the world simulation capacity of video diffusion models to trigger a new perpetual scene view generation task.
  arXiv  Detail & Related papers  (2025-06-21T12:51:34Z)
• WorldExplorer: Towards Generating Fully Navigable 3D Scenes [49.21733308718443]
  WorldExplorer builds fully navigable 3D scenes with consistent visual quality across a wide range of viewpoints.<n>We generate multiple videos along short, pre-defined trajectories, that explore the scene in depth.<n>Our novel scene memory conditions each video on the most relevant prior views, while a collision-detection mechanism prevents degenerate results.
  arXiv  Detail & Related papers  (2025-06-02T15:41:31Z)
• PaintScene4D: Consistent 4D Scene Generation from Text Prompts [29.075849524496707]
  PaintScene4D is a novel text-to-4D scene generation framework.<n>It harnesses video generative models trained on diverse real-world datasets.<n>It produces realistic 4D scenes that can be viewed from arbitrary trajectories.
  arXiv  Detail & Related papers  (2024-12-05T18:59:57Z)
• Forecasting Future Videos from Novel Views via Disentangled 3D Scene Representation [54.60804602905519]
  We learn an entangled representation, aiming to model layered scene geometry, motion forecasting and novel view synthesis together. Our approach chooses to disentangle scene geometry from scene motion, via lifting the 2D scene to 3D point clouds. To model future 3D scene motion, we propose a disentangled two-stage approach that initially forecasts ego-motion and subsequently the residual motion of dynamic objects.
  arXiv  Detail & Related papers  (2024-07-31T08:54:50Z)
• Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image [59.18564636990079]
  We study the problem of synthesizing a long-term dynamic video from only a single image. Existing methods either hallucinate inconsistent perpetual views or struggle with long camera trajectories. We present Make-It-4D, a novel method that can generate a consistent long-term dynamic video from a single image.
  arXiv  Detail & Related papers  (2023-08-20T12:53:50Z)
• Decoupling Dynamic Monocular Videos for Dynamic View Synthesis [50.93409250217699]
  We tackle the challenge of dynamic view synthesis from dynamic monocular videos in an unsupervised fashion. Specifically, we decouple the motion of the dynamic objects into object motion and camera motion, respectively regularized by proposed unsupervised surface consistency and patch-based multi-view constraints.
  arXiv  Detail & Related papers  (2023-04-04T11:25:44Z)
• 3D Cinemagraphy from a Single Image [73.09720823592092]
  We present 3D Cinemagraphy, a new technique that marries 2D image animation with 3D photography. Given a single still image as input, our goal is to generate a video that contains both visual content animation and camera motion.
  arXiv  Detail & Related papers  (2023-03-10T06:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.