ReWorld: An Interactive World Model with Long-Horizon Memory

• URL: http://arxiv.org/abs/2608.23565v1
• Date: Mon, 24 Aug 2026 17:59:05 GMT
• Title: ReWorld: An Interactive World Model with Long-Horizon Memory
• Abstract summary: ReWorld separates control and memory during training and bounds them at inference.<n>A metric-scale-aligned data engine places eight sources -- Unreal-rendered fly-throughs, game roaming, and real-world footage -- on one physical action scale.<n>One backbone serves both a high-fidelity multi-step mode and a real-time interactive one.
• Score: 43.24590497177396
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An interactive world model must follow the user's actions, remember the places it has shown, and stream in real time. The tension is structural: control wants a short horizon, memory wants an unbounded one. ReWorld separates the two during training and bounds them at inference. Mixed per-head attention windows confine most heads to the recent past while a small set of global heads attends over the entire history, and random head routing keeps either capability from binding to particular heads; random chunk dropping makes sparse histories in-distribution. At inference the whole past lives under a fixed budget: a bounded KV cache backed by a pose-indexed landmark bank, from which the model retrieves the landmarks nearest the current pose. A metric-scale-aligned data engine places eight sources -- Unreal-rendered fly-throughs, game roaming, and real-world footage -- on one physical action scale, so the same key press moves the camera the same distance in every source, and palindrome trajectories supply the revisit evidence that memory training needs. Distribution-matching distillation confined to a LoRA adapter then compresses sampling to four steps: one backbone serves both a high-fidelity multi-step mode and a real-time interactive one, streaming 704x1280 video across photorealistic, game-style, and stylized worlds. Under a three-axis protocol covering action following, long-horizon recall, and video quality, against six recent interactive world models it attains the best control fidelity ($11.95^\circ$ rotation error and the best camera-motion consistency) and the best generation quality; and on minute-long out-and-back rollouts ($64$\,s, $384$ latents), its fixed 12-chunk cache still regenerates the starting view -- at rollout lengths where a sliding window has long evicted the evidence and full-KV attention runs out of memory.

Related papers

• Sekai2: From World Exploration to Interactive World Modeling [22.57939715452233]
  Sekai2 is a multi-source real-world video dataset that carries the world-exploration footage of Sekai toward interactive world modeling.<n>The release contains 128,892 clips totaling 2,826 hours from 10,428 source videos across 113 countries or regions.<n>Every clip includes a released camera trajectory and hierarchical annotations disentangling subject motion, environment dynamics, static scene content, and camera behavior, resulting in 649,597 temporally grounded segments.
  arXiv  Detail & Related papers  (2026-08-10T11:23:01Z)
• Addressable Memory for Video World Models [50.40232922140905]
  We study visual persistence in interactive video world models.<n>We propose WorldTrace, a training-free memory framework for long-horizon visual persistence.
  arXiv  Detail & Related papers  (2026-08-07T16:55:57Z)
• Wonder: Video World Model Done Better [70.28870858365214]
  We present Wonder, a general-purpose video world model for real-time, camera-controllable world exploration.<n>Given an image or a conditional video, Wonder constructs a playable world where users can navigate interactively by moving the camera.<n>We introduce a novel camera conditioning with a dense coordinate field whose renderings provide spatially aligned motion and orientation cues.
  arXiv  Detail & Related papers  (2026-07-28T17:45:25Z)
• AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report [39.20781986625199]
  AlayaWorld is an interactive long-horizon video world model that generates 24-fps video at 540p and 720p.<n>Built on a 15B video diffusion transformer, AlayaWorld generates short latent chunks autoregressively under camera trajectories.<n>On iWorld-Bench, AlayaWorld achieves the best performance over long-horizon generation.
  arXiv  Detail & Related papers  (2026-07-20T17:15:41Z)
• EgoCS-400K: An Egocentric Gameplay Dataset for World Models [53.9389499966106]
  We introduce EgoCS-400K, a large-scale replay-grounded egocentric Counter-Strike dataset for world models.<n>It is built from public professional CS and CS2 match demos that preserve human gameplay trajectories.<n>EgoCS-400K contains over 400,000 first-person videos and 10,000 hours of gameplay from more than 1,000 matches and 40,000 rounds.
  arXiv  Detail & Related papers  (2026-06-16T17:13:58Z)
• ActWorld: From Explorable to Interactive World Model via Action-Aware Memory [36.88820961480639]
  We present ActWorld, an interactive world model that supports mid-rollout object interaction within a chunk-autoregressive framework.<n> Experiments show that ActWorld supports both flexible navigation and rich object interaction within a single model.
  arXiv  Detail & Related papers  (2026-06-16T09:47:32Z)
• WorldKV: Efficient World Memory with World Retrieval and Compression [56.57011243315561]
  We propose WorldKV, a training-free framework with two components: World Retrieval and World Compression.<n>On Matrix-Game-2.0 and LingBot- World-Fast, WorldKV matches or exceeds full- KV memory fidelity at roughly 2x the throughput, and is competitive with memory-trained baselines without any fine-tuning.
  arXiv  Detail & Related papers  (2026-05-21T16:55:04Z)
• WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling [34.486078065308995]
  WorldPlay is a streaming video diffusion model that enables real-time, interactive world modeling with long-term geometric consistency.<n>We use a Dual Action Representation to enable robust action control in response to the user's keyboard and mouse inputs.<n>We also propose Context Forcing, a novel distillation method designed for memory-aware model.
  arXiv  Detail & Related papers  (2025-12-16T17:22:46Z)
• RELIC: Interactive Video World Model with Long-Horizon Memory [74.81433479334821]
  A truly interactive world model requires real-time long-horizon streaming, consistent spatial memory, and precise user control.<n>We present RELIC, a unified framework that tackles these three challenges altogether.<n>Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time.
  arXiv  Detail & Related papers  (2025-12-03T18:29:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.