Walking in the Implicit: Interactive World Exploration via Neural Scene Representation

• URL: http://arxiv.org/abs/2606.30045v1
• Date: Mon, 29 Jun 2026 09:37:33 GMT
• Title: Walking in the Implicit: Interactive World Exploration via Neural Scene Representation
• Abstract summary: We propose Walking in the Implicit, a scene-centric paradigm that changes the rollout variable from frame latents to a fixed-length, renderable state.<n>We instantiate this paradigm as NeuWorld: a transformer VAE learns locally anchored NIS from sparse posed frames.<n>By reusing the VAE encoder as a unified conditioner, NeuWorld maps camera, reference-image, and history cues into the same NIS modality.
• Score: 18.76731897279849
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Interactive video generation systems for camera-controlled world exploration roll out growing sequences of latent video frames, entangling state transition with high-frequency observation synthesis. We propose Walking in the Implicit, a scene-centric paradigm that changes the rollout variable from frame latents to a fixed-length, renderable implicit state, termed Neural Implicit Scene (NIS). This factorizes interactive generation into stochastic transition of a compact scene state and deterministic pose-conditioned rendering given the sampled state. We instantiate this paradigm as NeuWorld: a transformer VAE learns locally anchored NIS from sparse posed frames, and a diffusion transformer evolves NIS conditioned on future camera trajectories and geometry-aware retrieved history. By reusing the VAE encoder as a unified conditioner, NeuWorld maps camera, reference-image, and history cues into the same NIS modality, avoiding external heterogeneous encoders. Trained from scratch on public posed-view data without pretrained video backbones or auxiliary 3D reconstructors, NeuWorld achieves strong long-horizon consistency with favorable inference efficiency.

Related papers

• DiVA: Enabling Interactive Digital Life Simulation via Video Models [107.640138694388]
  DiVA is an interactive digital life simulator pioneering a new paradigm for long-term, open-ended interactive experiences.<n>It maintains long-term visual quality and realism, validating its effectiveness as a sustainable, interactive simulation.
  arXiv  Detail & Related papers  (2026-09-12T09:33:15Z)
• MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data [125.43597497646444]
  MetaWorld is a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos.<n>It achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.
  arXiv  Detail & Related papers  (2026-06-01T18:20:20Z)
• RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space [51.441415833480505]
  RAYNOVA is a multiview world model for driving scenarios that employs a dual-causal autoregressive framework.<n>It constructs an isotropic-temporal representation across views, frames, and scales based on relative Plcker-ray positional encoding.
  arXiv  Detail & Related papers  (2026-02-24T08:41:40Z)
• RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusion [21.761449995572757]
  RecurGS is a recurrent fusion framework that integrates discrete Gaussian scene states into a single evolving representation.<n>A voxelized, visibility-aware fusion module selectively incorporates newly observed regions while keeping stable areas fixed.<n>Our framework delivers high-quality reconstructions with substantially improved update efficiency.
  arXiv  Detail & Related papers  (2025-12-20T14:53:22Z)
• DT-NVS: Diffusion Transformers for Novel View Synthesis [22.458328201080715]
  We propose a 3D-aware diffusion model for generalized novel view synthesis.<n>We make significant contributions to transformer and self-attention architectures to translate images to 3d representations.<n>We show improvements over state-of-the-art 3D aware diffusion models and deterministic approaches.
  arXiv  Detail & Related papers  (2025-11-11T22:40:00Z)
• ARSS: Taming Decoder-only Autoregressive Visual Generation for View Synthesis From Single View [11.346049532150127]
  textbfARSS is a framework that generates novel views from a single image conditioned on a camera trajectory.<n>Our method performs comparably to, or better than, state-of-the-art view synthesis approaches based on diffusion models.
  arXiv  Detail & Related papers  (2025-09-27T00:03:09Z)
• 4D Driving Scene Generation With Stereo Forcing [62.47705572424127]
  Current generative models struggle to synthesize dynamic 4D driving scenes that simultaneously support temporal extrapolation and spatial novel view synthesis (NVS) without per-scene optimization.<n>We present PhiGenesis, a unified framework for 4D scene generation that extends video generation techniques with geometric and temporal consistency.
  arXiv  Detail & Related papers  (2025-09-24T15:37:17Z)
• A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wild [72.0226493284814]
  We propose a unified framework for event-based frame that performs deblurring ad-hoc.<n>Our network consistently outperforms previous state-of-the-art methods on frame, single image deblurring, and the joint task of both.
  arXiv  Detail & Related papers  (2023-01-12T18:19:00Z)
• Video Frame Interpolation with Transformer [55.12620857638253]
  We introduce a novel framework, which takes advantage of Transformer to model long-range pixel correlation among video frames. Our network is equipped with a novel cross-scale window-based attention mechanism, where cross-scale windows interact with each other.
  arXiv  Detail & Related papers  (2022-05-15T09:30:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.