Geometry-Aware Implicit Memory for Video World Models

• URL: http://arxiv.org/abs/2606.02436v1
• Date: Mon, 01 Jun 2026 16:08:43 GMT
• Title: Geometry-Aware Implicit Memory for Video World Models
• Abstract summary: GIM-World is a geometry-aware implicit memory framework for video world models.<n>A camera-queryable geometry head distills 3D scene structure from a frozen foundation model into the memory during training.<n>Experiments on MIND show that GIM-World better preserves long-horizon geometric and visual consistency than both explicit- and implicit-memory baselines.
• Score: 29.522291951915577
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video world models aim to simulate controllable visual environments, but long-horizon rollouts depend on what the model remembers after observations leave its native context window. Explicit memories retain frames or online 3D reconstructions, which can suffer from heuristic retrieval errors, redundant appearance storage, or reconstruction artifacts. Implicit memories compress history into a compact state, but existing designs are not explicitly constrained to encode cross-view scene geometry. We propose GIM-World, a geometry-aware implicit memory framework for video world models. A lightweight transformer encoder compresses variable-length history into fixed-size memory tokens, a camera-queryable geometry head distills 3D scene structure from a frozen foundation model into the memory during training, and an information-guided pruning rule keeps encoding cost bounded as history grows. The geometry teacher is discarded at inference, leaving a lightweight memory module. Experiments on MIND show that GIM-World better preserves long-horizon geometric and visual consistency than both explicit- and implicit-memory baselines.

Related papers

• PermaVid: Consistent Video Generation Across Edits via Disentangled Context Memory [80.58558679163197]
  PermaVid is a novel framework built upon a multi-modal context memory that disentangles spatial context into semantic appearance and geometric structure.<n>Our method maintains strong long-term semantic and structural consistency after edits, significantly outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-06-15T09:20:32Z)
• Latent Spatial Memory for Video World Models [45.02259712072663]
  We introduce emphlatent spatial memory for video world models, a persistent 3D cache that stores scene information directly in the diffusion latent space, avoiding pixel-space reconstruction.<n>Experiments show that latent spatial memory achieves up to textbf10.57$times$ faster end-to-end video generation and textbf55$times$ reduction in memory footprint relative to explicit 3D baselines.
  arXiv  Detail & Related papers  (2026-06-08T17:59:54Z)
• Echo-Memory: A Controlled Study of Memory in Action World Models [58.77172260403133]
  We present a controlled study of memory mechanisms in action-conditioned world models.<n>Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences.<n>We use Echo-Memory to fix the action-to-video interface and varies only how history is stored and read by the generator.
  arXiv  Detail & Related papers  (2026-06-08T17:54:10Z)
• AnyRecon: Arbitrary-View 3D Reconstruction with Video Diffusion Model [29.087865646035695]
  Sparse-view 3D reconstruction is essential for modeling scenes from casual captures, but remain challenging for non-generative reconstruction.<n>We propose AnyRecon, a scalable framework for reconstruction from arbitrary and unordered sparse inputs.<n>Experiments demonstrate robust and scalable reconstruction across irregular inputs, large viewpoint gaps, and long trajectories.
  arXiv  Detail & Related papers  (2026-04-21T17:59:47Z)
• I3DM: Implicit 3D-aware Memory Retrieval and Injection for Consistent Video Scene Generation [56.33710337846449]
  I3DM is a novel implicit 3D-aware memory mechanism for consistent video scene generation.<n>At the core of our approach is a 3D-aware memory retrieval strategy.<n>To fully utilize the retrieved historical frames, we introduce a 3D-aligned memory injection module.
  arXiv  Detail & Related papers  (2026-03-24T16:45:40Z)
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories [78.78355829813793]
  Existing memory-based approaches often condition generation on globally reconstructed 3D scenes by rendering anchor videos from the reconstructed geometry in the history.<n>We introduce AnchorWeave, a memory-augmented video generation framework that replaces a single misaligned global memory with multiple clean local geometric memories.<n>Experiments demonstrate that AnchorWeave significantly improves long-term scene consistency while maintaining strong visual quality.
  arXiv  Detail & Related papers  (2026-02-16T17:23:08Z)
• Spatia: Video Generation with Updatable Spatial Memory [60.21619361473996]
  Spatia is a spatial memory-aware video generation framework that preserves a 3D scene point cloud as persistent spatial memory.<n>Spatia iteratively generates video clips conditioned on this spatial memory and continuously updates it through visual SLAM.<n>Spatia enables applications such as explicit camera control and 3D-aware interactive editing, providing a geometrically grounded framework for scalable, memory-driven video generation.
  arXiv  Detail & Related papers  (2025-12-17T18:59:59Z)
• RELIC: Interactive Video World Model with Long-Horizon Memory [74.81433479334821]
  A truly interactive world model requires real-time long-horizon streaming, consistent spatial memory, and precise user control.<n>We present RELIC, a unified framework that tackles these three challenges altogether.<n>Given a single image and a text description, RELIC enables memory-aware, long-duration exploration of arbitrary scenes in real time.
  arXiv  Detail & Related papers  (2025-12-03T18:29:20Z)
• VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory [55.73900731190389]
  We introduce Surfel-Indexed View Memory (VMem), a memory module that remembers past views by indexing them geometrically based on the 3D surface elements (surfels) they have observed.<n>VMem enables efficient retrieval of the most relevant past views when generating new ones.
  arXiv  Detail & Related papers  (2025-06-23T17:59:56Z)
• Video World Models with Long-term Spatial Memory [110.530715838396]
  We introduce a novel framework to enhance long-term consistency of video world models.<n>Our framework includes mechanisms to store and retrieve information from the long-term spatial memory.<n>Our evaluations show improved quality, consistency, and context length compared to relevant baselines.
  arXiv  Detail & Related papers  (2025-06-05T17:42:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.