InSpatio-WorldFM: An Open-Source Real-Time Generative Frame Model

• URL: http://arxiv.org/abs/2603.11911v1
• Date: Thu, 12 Mar 2026 13:28:50 GMT
• Title: InSpatio-WorldFM: An Open-Source Real-Time Generative Frame Model
• Abstract summary: We present InSpatio-WorldFM, an open-source real-time frame model for spatial intelligence.<n>InSpatio-WorldFM adopts a frame-based paradigm that generates each frame independently, enabling low-latency real-time spatial inference.
• Score: 17.421965971185134
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present InSpatio-WorldFM, an open-source real-time frame model for spatial intelligence. Unlike video-based world models that rely on sequential frame generation and incur substantial latency due to window-level processing, InSpatio-WorldFM adopts a frame-based paradigm that generates each frame independently, enabling low-latency real-time spatial inference. By enforcing multi-view spatial consistency through explicit 3D anchors and implicit spatial memory, the model preserves global scene geometry while maintaining fine-grained visual details across viewpoint changes. We further introduce a progressive three-stage training pipeline that transforms a pretrained image diffusion model into a controllable frame model and finally into a real-time generator through few-step distillation. Experimental results show that InSpatio-WorldFM achieves strong multi-view consistency while supporting interactive exploration on consumer-grade GPUs, providing an efficient alternative to traditional video-based world models for real-time world simulation.

Related papers

• World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoning [18.658825757342544]
  Vision-language models (VLMs) have shown strong performance on static visual understanding.<n>They still struggle with dynamic spatial reasoning that requires imagining how scenes evolve under egocentric motion.<n>We propose World2VLM, a training framework that distills spatial imagination from a generative world model into a vision-language model.
  arXiv  Detail & Related papers  (2026-04-29T17:48:01Z)
• Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory [53.39687409541093]
  Matrix-Game 3.0 is a memory-augmented interactive world model designed for 720p real-time longform video generation.<n>We introduce systematic improvements across data, model, and inference.<n> Experimental results show that Matrix-Game 3.0 achieves up to 40 FPS real-time generation at 720p resolution with a 5B model.
  arXiv  Detail & Related papers  (2026-04-10T06:00:09Z)
• RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space [51.441415833480505]
  RAYNOVA is a multiview world model for driving scenarios that employs a dual-causal autoregressive framework.<n>It constructs an isotropic-temporal representation across views, frames, and scales based on relative Plcker-ray positional encoding.
  arXiv  Detail & Related papers  (2026-02-24T08:41:40Z)
• TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model [53.555353366322464]
  We present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system.<n>Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible synthesis systems.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• Learning Spatio-Temporal Feature Representations for Video-Based Gaze Estimation [50.05866669110754]
  Video-based gaze estimation methods aim to capture the inherently temporal dynamics of human eye gaze from multiple image frames.<n>We propose the Spatio-Temporal Gaze Network (ST-Gaze), a model that combines a CNN backbone with dedicated channel attention and self-attention modules.<n>We show that ST-Gaze achieves state-of-the-art performance both with and without person-specific adaptation.
  arXiv  Detail & Related papers  (2025-12-19T15:15:58Z)
• DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion [53.70278210626701]
  We propose a data-driven multi-view reasoning approach that directly infers 3D scene geometry and camera poses from multi-view images.<n>Our framework, DiffusionSfM, parameterizes scene geometry and cameras as pixel-wise ray origins and endpoints in a global frame.<n>We empirically validate DiffusionSfM on both synthetic and real datasets, demonstrating that it outperforms classical and learning-based approaches.
  arXiv  Detail & Related papers  (2025-05-08T17:59:47Z)
• Seeing World Dynamics in a Nutshell [132.79736435144403]
  NutWorld is a framework that transforms monocular videos into dynamic 3D representations in a single forward pass.<n>We demonstrate that NutWorld achieves high-fidelity video reconstruction quality while enabling downstream applications in real-time.
  arXiv  Detail & Related papers  (2025-02-05T18:59:52Z)
• EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling [8.250616459360684]
  We introduce EDELINE, a unified world model architecture that integrates state space models with diffusion models.<n>Our approach outperforms existing baselines across visually challenging Atari 100k tasks, memory-demanding benchmark, and 3D first-person ViZDoom environments.
  arXiv  Detail & Related papers  (2025-02-01T15:49:59Z)
• FACTS: A Factored State-Space Framework For World Modelling [24.08175276756845]
  We propose a novel recurrent framework, the textbfFACTored textbfState-space (textbfFACTS) model, for spatial-temporal world modelling.<n>The FACTS framework constructs a graph-memory with a routing mechanism that learns permutable memory representations.<n>It consistently outperforms or matches specialised state-of-the-art models, despite its general-purpose world modelling design.
  arXiv  Detail & Related papers  (2024-10-28T11:04:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.