AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report

• URL: http://arxiv.org/abs/2607.18367v1
• Date: Mon, 20 Jul 2026 17:15:41 GMT
• Title: AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report
• Abstract summary: AlayaWorld is an interactive long-horizon video world model that generates 24-fps video at 540p and 720p.<n>Built on a 15B video diffusion transformer, AlayaWorld generates short latent chunks autoregressively under camera trajectories.<n>On iWorld-Bench, AlayaWorld achieves the best performance over long-horizon generation.
• Score: 39.20781986625199
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unlike conventional video game development, which relies on labor-intensive pipelines for asset production, animation, physics, and programming, video world models generate interactive environments from user inputs instantly. It enable us to create customized, explorable, and continuously evolving virtual world from text, an image, or video. Realizing this vision requires four tightly coupled capabilities: interaction, persistent spatiotemporal consistency, stable long-horizon generation, and efficient response. We present AlayaWorld, an interactive long-horizon video world model that generates 24-fps video at 540p and 720p. Built on a 15B video diffusion transformer, AlayaWorld generates short latent chunks autoregressively under camera trajectories and switchable text prompts. Its bounded visual context combines a persistent sink frame, compressed temporal history, geometry-aligned spatial memory, and recent-frame conditioning. To reduce long-term drift, the model is trained with corrupted histories and prediction residuals collected from its own roll-outs. We further introduce a discrete autoregressive distillation formulation that combines distribution-matching distillation, self-forcing++, and consistency distillation, reducing inference from approximately 30 sampling steps to four steps per chunk. On iWorld-Bench, AlayaWorld achieves the best performance over long-horizon generation. Conceived as a full-stack, open-source, and long-term project, AlayaWorld is intended to provide an extensible foundation for future research on interactive video world models.

Related papers

• Wonder: Video World Model Done Better [70.28870858365214]
  We present Wonder, a general-purpose video world model for real-time, camera-controllable world exploration.<n>Given an image or a conditional video, Wonder constructs a playable world where users can navigate interactively by moving the camera.<n>We introduce a novel camera conditioning with a dense coordinate field whose renderings provide spatially aligned motion and orientation cues.
  arXiv  Detail & Related papers  (2026-07-28T17:45:25Z)
• AlayaWorld: Long-Horizon and Playable Video World Generation [36.52656377442462]
  We present textbfAlayaWorld, a full-stack open-source framework for building interactive generative worlds.<n>AlayaWorld enables open-ended real-time interaction, allowing users to freely navigate and perform diverse actions such as combat, spell casting, and monster summoning.
  arXiv  Detail & Related papers  (2026-07-07T13:59:57Z)
• Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory [53.39687409541093]
  Matrix-Game 3.0 is a memory-augmented interactive world model designed for 720p real-time longform video generation.<n>We introduce systematic improvements across data, model, and inference.<n> Experimental results show that Matrix-Game 3.0 achieves up to 40 FPS real-time generation at 720p resolution with a 5B model.
  arXiv  Detail & Related papers  (2026-04-10T06:00:09Z)
• LivingWorld: Interactive 4D World Generation with Environmental Dynamics [8.868060488503847]
  We introduce LivingWorld, an interactive framework for generating 4D worlds with environmental dynamics from a single image.<n>LivingWorld addresses this challenge by progressively constructing a globally coherent motion field as the scene expands.<n>We further represent motion using a compact hash-based motion field, enabling efficient querying and stable propagation of dynamics throughout the scene.
  arXiv  Detail & Related papers  (2026-04-02T05:38:48Z)
• TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model [53.555353366322464]
  We present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system.<n>Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible synthesis systems.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• Endless World: Real-Time 3D-Aware Long Video Generation [57.411689597435334]
  Endless World is a real-time framework for infinite, 3D-consistent video generation.<n>We introduce a conditional autoregressive training strategy that aligns newly generated content with existing video frames.<n>Our 3D injection mechanism enforces physical plausibility and geometric consistency throughout extended sequences.
  arXiv  Detail & Related papers  (2025-12-13T19:06:12Z)
• RELIC: Interactive Video World Model with Long-Horizon Memory [74.81433479334821]
  A truly interactive world model requires real-time long-horizon streaming, consistent spatial memory, and precise user control.<n>We present RELIC, a unified framework that tackles these three challenges altogether.<n>Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time.
  arXiv  Detail & Related papers  (2025-12-03T18:29:20Z)
• Matrix-Game 2.0: An Open-Source, Real-Time, and Streaming Interactive World Model [15.16063778402193]
  Matrix-Game 2.0 is an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion.<n>It can generate high-quality minute-level videos across diverse scenes at an ultra-fast speed of 25 FPS.
  arXiv  Detail & Related papers  (2025-08-18T15:28:53Z)
• Video World Models with Long-term Spatial Memory [110.530715838396]
  We introduce a novel framework to enhance long-term consistency of video world models.<n>Our framework includes mechanisms to store and retrieve information from the long-term spatial memory.<n>Our evaluations show improved quality, consistency, and context length compared to relevant baselines.
  arXiv  Detail & Related papers  (2025-06-05T17:42:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.