WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation

• URL: http://arxiv.org/abs/2603.16871v1
• Date: Tue, 17 Mar 2026 17:59:56 GMT
• Title: WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation
• Abstract summary: We establish camera pose as a unifying geometric representation to ground immediate action control and long-term 3D consistency.<n>Our approach substantially outperforms state-of-the-art interactive gaming world models in action controllability, long-horizon visual quality, and 3D spatial consistency.
• Score: 47.97929550105451
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in video diffusion transformers have enabled interactive gaming world models that allow users to explore generated environments over extended horizons. However, existing approaches struggle with precise action control and long-horizon 3D consistency. Most prior works treat user actions as abstract conditioning signals, overlooking the fundamental geometric coupling between actions and the 3D world, whereby actions induce relative camera motions that accumulate into a global camera pose within a 3D world. In this paper, we establish camera pose as a unifying geometric representation to jointly ground immediate action control and long-term 3D consistency. First, we define a physics-based continuous action space and represent user inputs in the Lie algebra to derive precise 6-DoF camera poses, which are injected into the generative model via a camera embedder to ensure accurate action alignment. Second, we use global camera poses as spatial indices to retrieve relevant past observations, enabling geometrically consistent revisiting of locations during long-horizon navigation. To support this research, we introduce a large-scale dataset comprising 3,000 minutes of authentic human gameplay annotated with camera trajectories and textual descriptions. Extensive experiments show that our approach substantially outperforms state-of-the-art interactive gaming world models in action controllability, long-horizon visual quality, and 3D spatial consistency.

Related papers

• Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory [28.887112448832525]
  We present Matrix-Game 3.5, which advances real-time interactive world generation through three key improvements.<n>First, we propose a unified geometry-aware memory framework, whose patch-memory and tiled-PRoPE components introduce no additional learnable parameters.<n>Second, we introduce a static-dynamic disentangled world representation that separately models static scene geometry and dynamic subjects.<n>Third, we develop a two-stage progressive real-time distillation framework that converts a bidirectional diffusion model into a few-step causal generator.
  arXiv  Detail & Related papers  (2026-08-30T17:11:00Z)
• SpatialDiff: 3D-Aware Object Movement via Implicit Spatial Modeling [60.4038189368113]
  We introduce Implicit 3D Spatial Modeling, which introduces 3D prior knowledge and enables the model to internally build a comprehensive understanding of the three-dimensional spatial structure.<n>We also present Global Spatial Supervision, which constrains the latent spatial features to enable the model to perceive changes in object spatial positions caused by editing operations.<n> Experimental results demonstrate that our method significantly improves the accuracy and fidelity of spatial movement in complex scenes.
  arXiv  Detail & Related papers  (2026-08-22T04:46:08Z)
• GMT: Goal-Conditioned Multimodal Transformer for 6-DOF Object Trajectory Synthesis in 3D Scenes [47.88691731631585]
  GMT is a multimodal transformer framework that generates realistic and goal-directed object trajectories.<n>Experiments on synthetic and real-world benchmarks demonstrate that GMT outperforms state-of-the-art human motion and human-object interaction baselines.
  arXiv  Detail & Related papers  (2026-03-18T17:54:35Z)
• Beyond Pixel Histories: World Models with Persistent 3D State [50.4601060508243]
  PERSIST is a new paradigm of world model which simulates the evolution of a latent 3D scene.<n>We show substantial improvements in spatial memory, 3D consistency, and long-horizon stability over existing methods.
  arXiv  Detail & Related papers  (2026-03-03T19:58:31Z)
• Hand2World: Autoregressive Egocentric Interaction Generation via Free-Space Hand Gestures [33.2764643227486]
  Egocentric interactive world models are essential for augmented reality and embodied AI, where visual generation must respond to user input with low latency, geometric consistency, and long-term stability.<n>We study egocentric interaction generation from a single scene image under free-space hand gestures, aiming to synthesize photorealistic videos in which hands enter the scene, interact with objects, and induce plausible world dynamics under head motion.<n>This setting introduces fundamental challenges, including distribution shift between free-space gestures and contact-heavy training data, ambiguity between hand motion and camera motion in monocular views, and the need for arbitrary-length video generation.
  arXiv  Detail & Related papers  (2026-02-10T09:51:07Z)
• Geometry-Aware Rotary Position Embedding for Consistent Video World Model [48.914346802616414]
  ViewRope is a geometry-aware encoding that injects camera-ray directions directly into video transformer self-attention layers.<n>Geometry-Aware Frame-Sparse Attention exploits these geometric cues to selectively attend to relevant historical frames.<n>Our results demonstrate that ViewRope substantially improves long-term consistency while reducing computational costs.
  arXiv  Detail & Related papers  (2026-02-08T08:01:16Z)
• EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory [40.346684158976494]
  EvoWorld bridges panoramic video generation with evolving 3D memory to enable spatially consistent long-horizon exploration.<n>Unlike prior state-of-the-arts that synthesize videos only, our key insight lies in exploiting this evolving 3D reconstruction as explicit spatial guidance.<n>To evaluate long-range exploration capabilities, we introduce the first comprehensive benchmark spanning synthetic outdoor environments, Habitat indoor scenes, and challenging real-world scenarios.
  arXiv  Detail & Related papers  (2025-10-01T17:59:38Z)
• Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation [66.95956271144982]
  We present Voyager, a novel video diffusion framework that generates world-consistent 3D point-cloud sequences from a single image.<n>Unlike existing approaches, Voyager achieves end-to-end scene generation and reconstruction with inherent consistency across frames.
  arXiv  Detail & Related papers  (2025-06-04T17:59:04Z)
• MagicDrive: Street View Generation with Diverse 3D Geometry Control [82.69871576797166]
  We introduce MagicDrive, a novel street view generation framework, offering diverse 3D geometry controls. Our design incorporates a cross-view attention module, ensuring consistency across multiple camera views.
  arXiv  Detail & Related papers  (2023-10-04T06:14:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.