MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold

• URL: http://arxiv.org/abs/2606.13376v2
• Date: Wed, 17 Jun 2026 16:38:12 GMT
• Title: MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold
• Abstract summary: We present MoVerse, a real-time video world model that creates an interactively navigable scene from a single narrow-field-of-view image.<n>MoVerse supports real-time scene roaming at 8FPS on a single NVIDIA4090 GPU.
• Score: 45.96225433489696
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present MoVerse, a real-time video world model that creates an interactively navigable scene from a single narrow-field-of-view image. This setting is challenging because the input observes only a small fraction of the environment, while interactive roaming requires a complete surrounding world, persistent geometry, controllable camera motion, and temporally coherent high-fidelity observations. MoVerse addresses this problem by separating world construction from observation rendering. It first expands the input into a gravity-aligned 360$^\circ$ panorama with topology-aware diffusion, closing the missing field of view before 3D reasoning. It then lifts the panorama into a persistent 3D Gaussian scaffold using panoramic geometry-aware residual prediction, yielding a dense and directly renderable spatial memory. Finally, a Gaussian-conditioned video renderer translates scaffold renderings along user-specified camera trajectories into photorealistic video. To make this renderer practical for interaction, we train a bidirectional diffusion teacher for high-quality conditional rendering and distill it into a causal autoregressive student for bounded-latency streaming. This design combines the controllability and long-range consistency of explicit 3D representations with the perceptual quality of generative video models. MoVerse supports real-time scene roaming at 8~FPS on a single NVIDIA RTX~4090 GPU, demonstrating a practical path toward single-image world creation with interactive video output.

Related papers

• Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulation [9.228046414885494]
  Existing methods either lack metric trajectory control, which hinders reliable downstream 3D reconstruction, or struggle with large disocclusions under long-range camera motion.<n>We present Genie Sim PanoWorld, a two-stage feed-forward pipeline that bridges generation and reconstruction via an explicit, trajectory-controllable panoramic video.
  arXiv  Detail & Related papers  (2026-07-29T09:08:16Z)
• ABot-3DWorld 0: A Universal World Model to Explore Any 3D Space [49.372892542771886]
  ABot-3DWorld 0 is a universal multimodal 3D world model that turns text, image, and video inputs into high-fidelity, explorable 3D worlds.
  arXiv  Detail & Related papers  (2026-07-13T15:14:48Z)
• Rein3D: Reinforced 3D Indoor Scene Generation with Panoramic Video Diffusion Models [38.45163624089566]
  Rein3D is a framework that reconstructs full 360-degree indoor environments.<n>We employ a radial exploration strategy to render imperfect panoramic videos.<n>These sequences are restored by a panoramic video-to-video diffusion model and further enhanced via video super-resolution to synthesize high-fidelity geometry and textures.
  arXiv  Detail & Related papers  (2026-04-12T10:55:14Z)
• Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation [66.95956271144982]
  We present Voyager, a novel video diffusion framework that generates world-consistent 3D point-cloud sequences from a single image.<n>Unlike existing approaches, Voyager achieves end-to-end scene generation and reconstruction with inherent consistency across frames.
  arXiv  Detail & Related papers  (2025-06-04T17:59:04Z)
• WorldExplorer: Towards Generating Fully Navigable 3D Scenes [48.16064304951891]
  WorldExplorer builds fully navigable 3D scenes with consistent visual quality across a wide range of viewpoints.<n>We generate multiple videos along short, pre-defined trajectories, that explore the scene in depth.<n>Our novel scene memory conditions each video on the most relevant prior views, while a collision-detection mechanism prevents degenerate results.
  arXiv  Detail & Related papers  (2025-06-02T15:41:31Z)
• Seeing World Dynamics in a Nutshell [132.79736435144403]
  NutWorld is a framework that transforms monocular videos into dynamic 3D representations in a single forward pass.<n>We demonstrate that NutWorld achieves high-fidelity video reconstruction quality while enabling downstream applications in real-time.
  arXiv  Detail & Related papers  (2025-02-05T18:59:52Z)
• Persistent Nature: A Generative Model of Unbounded 3D Worlds [74.51149070418002]
  We present an extendable, planar scene layout grid that can be rendered from arbitrary camera poses via a 3D decoder and volume rendering. Based on this representation, we learn a generative world model solely from single-view internet photos. Our approach enables scene extrapolation beyond the fixed bounds of current 3D generative models, while also supporting a persistent, camera-independent world representation.
  arXiv  Detail & Related papers  (2023-03-23T17:59:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.