AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)

• URL: http://arxiv.org/abs/2608.13492v1
• Date: Thu, 13 Aug 2026 17:21:03 GMT
• Title: AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)
• Abstract summary: We revise how conditioning signals are represented and integrated into the model.<n>New design is guided by a simple principle: conditioning signals should match the generated content as closely as possible in both latent representation and temporal structure.
• Score: 39.20781986625199
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This report presents an improved version of AlayaWorld. While the backbone architecture, chunk-wise autoregressive generation scheme, and training data remain unchanged from the previous release, we substantially revise how conditioning signals are represented and integrated into the model. The new design is guided by a simple principle: conditioning signals should match the generated content as closely as possible in both latent representation and temporal structure. To this end, we make two major changes. First, we replace the previous depth-warping-based spatial memory with a streaming 3D point-cache renderer. Second, we redesign the conditioning pipeline so that visual conditions are encoded in the same causal-VAE latent space, with temporal statistics consistent with those of the generated video. Concretely, the new version introduces six modifications: (1) replacing static-frame image conditioning with motion-aware latent conditioning; (2) causally encoding re-rendered spatial memory as a continuous sequence; (3) aligning the temporal-memory window in pixel space; (4) adopting hard memory dropout that removes memory tokens rather than zeroing them; (5) unifying the VAE encoding and decoding protocol across training and inference; and (6) removing the camera AdaLN branch, such that viewpoint control is provided entirely through the re-rendered spatial condition.

Related papers

• Latent Spatial Memory for Video World Models [45.02259712072663]
  We introduce emphlatent spatial memory for video world models, a persistent 3D cache that stores scene information directly in the diffusion latent space, avoiding pixel-space reconstruction.<n>Experiments show that latent spatial memory achieves up to textbf10.57$times$ faster end-to-end video generation and textbf55$times$ reduction in memory footprint relative to explicit 3D baselines.
  arXiv  Detail & Related papers  (2026-06-08T17:59:54Z)
• Echo-Memory: A Controlled Study of Memory in Action World Models [58.77172260403133]
  We present a controlled study of memory mechanisms in action-conditioned world models.<n>Existing memory designs are hard to compare because gains are entangled with backbone, training, retrieval, and evaluation differences.<n>We use Echo-Memory to fix the action-to-video interface and varies only how history is stored and read by the generator.
  arXiv  Detail & Related papers  (2026-06-08T17:54:10Z)
• Robust Dreamer: Deviation-Aware Latent Gaussian Memory for Action-Controlled AR Video Generation [89.70897512515477]
  Action-controlled image-to-video generation is a promising paradigm for interactive world simulation, where each control signal should elicit an immediate visual response.<n>Maintaining visual fidelity and 3D consistency over long autoregressive rollouts remains challenging.<n>Existing 3D-aware methods often suffer from catastrophic drift due to two impediments: information loss from textitLatent--RGB Cycling, and the training--inference gap induced by the textiterror-free hypothesis.<n>We present textbfRobust Dreamer, a memory-augmented framework built around how to
  arXiv  Detail & Related papers  (2026-05-29T05:21:33Z)
• Rethinking Memory as Continuously Evolving Connectivity [63.58181883607843]
  Existing memory-augmented LLM agents treat memory as a static repository with pre-defined representations and fixed retrieval pipelines.<n>We propose FluxMem, a connectivity-evolving memory framework that models memory as a heterogeneous graph and progressively refines its topology through three stages.<n>Across three fundamentally distinct benchmarks including LoCoMo, Mind2Web, and GAIA, FluxMem achieves consistent state-of-the-art performance, demonstrating strong adaptation and generalization in complex agentic environments.
  arXiv  Detail & Related papers  (2026-05-27T17:35:34Z)
• Attend Locally, Remember Linearly: Linear Attention as Cross-Frame Memory for Autoregressive Video Diffusion [61.57938553036056]
  ARL2 is a hybrid attention module that replaces quadratic cross-frame attention with a fixed-size recurrent state.<n>We decompose self-attention into two branches: an intra-frame softmax branch for spatial detail and local dependencies, and an inter-frame recurrent linear branch that maintains a fixed-size state for streaming context.<n>With 75% of layers replaced by hybrid linear attention, the model achieves up to 2.26 wall-clock speedup and 54% memory reduction, while maintaining comparable quality with improving temporal consistency.
  arXiv  Detail & Related papers  (2026-05-15T19:33:45Z)
• Ray-Aware Pointer Memory with Adaptive Updates for Streaming 3D Reconstruction [19.132257919770947]
  3D reconstruction from continuous image streams requires both accurate geometric aggregation and stable memory management.<n>We propose a ray-aware pointer memory for streaming 3D reconstruction that explicitly models both spatial location and viewing direction.<n>Our approach provides a principled framework for scalable and drift-resistant online 3D reconstruction from image streams.
  arXiv  Detail & Related papers  (2026-05-07T06:42:11Z)
• AnyRecon: Arbitrary-View 3D Reconstruction with Video Diffusion Model [29.087865646035695]
  Sparse-view 3D reconstruction is essential for modeling scenes from casual captures, but remain challenging for non-generative reconstruction.<n>We propose AnyRecon, a scalable framework for reconstruction from arbitrary and unordered sparse inputs.<n>Experiments demonstrate robust and scalable reconstruction across irregular inputs, large viewpoint gaps, and long trajectories.
  arXiv  Detail & Related papers  (2026-04-21T17:59:47Z)
• I3DM: Implicit 3D-aware Memory Retrieval and Injection for Consistent Video Scene Generation [56.33710337846449]
  I3DM is a novel implicit 3D-aware memory mechanism for consistent video scene generation.<n>At the core of our approach is a 3D-aware memory retrieval strategy.<n>To fully utilize the retrieved historical frames, we introduce a 3D-aligned memory injection module.
  arXiv  Detail & Related papers  (2026-03-24T16:45:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.