World Machine: Towards Generative World Modeling for Time-Series

• URL: http://arxiv.org/abs/2605.23025v1
• Date: Thu, 21 May 2026 20:48:51 GMT
• Title: World Machine: Towards Generative World Modeling for Time-Series
• Abstract summary: We present World Machine, a generative world-modeling architecture for time series.<n>It is a transformer-based architecture with latent states that enables adaptation to different amounts of observed data and contexts.
• Score: 37.99105200271773
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models represent a paradigm shift in generative AI, pursuing predictive understanding and controllable simulation of environments in a structured and generalizable way. We present World Machine, a generative world-modeling architecture for time series. It is a transformer-based architecture with latent states that enables adaptation to different amounts of observed data and contexts. This shows an improvement over traditional transformers, which have a computational and memory cost that scales quadratically with the context. Experiments on a proposed synthetic dataset, Toy1D, validate the approach's feasibility, demonstrate capabilities not found in conventional transformers, and highlight the contributions of each component of the training protocol.

Related papers

• From monoliths to modules: Decomposing transducers for efficient world modelling [74.41506965793417]
  We develop a framework for decomposing complex world models represented by transducers.<n>Our results clarify how to invert this process, deriving sub-transducers operating on distinct input-output subspaces.
  arXiv  Detail & Related papers  (2025-12-01T20:37:43Z)
• Learning Transformer-based World Models with Contrastive Predictive Coding [58.0159270859475]
  We show that the next state prediction objective is insufficient to fully exploit the representation capabilities of Transformers.<n>We propose to extend world model predictions to longer time horizons by introducing TWISTER, a world model using action-conditioned Contrastive Predictive Coding.<n>TWISTER achieves a human-normalized mean score of 162% on the Atari 100k benchmark, setting a new record among state-of-the-art methods that do not employ look-ahead search.
  arXiv  Detail & Related papers  (2025-03-06T13:18:37Z)
• Trajectory World Models for Heterogeneous Environments [67.27233466954814]
  Heterogeneity in sensors and actuators across environments poses a significant challenge to building large-scale pre-trained world models.<n>We introduce UniTraj, a unified dataset comprising over one million trajectories from 80 environments, designed to scale data while preserving critical diversity.<n>We also propose TrajWorld, a novel architecture capable of handling varying sensor and actuator information and capturing environment dynamics in-context.
  arXiv  Detail & Related papers  (2025-02-03T13:59:08Z)
• CENTS: Generating synthetic electricity consumption time series for rare and unseen scenarios [4.997673761305336]
  We propose a method for creating high-fidelity electricity consumption time series data for rare and unseen context variables.<n>Our results highlight the efficacy of the proposed method in generating realistic household-level electricity consumption data.
  arXiv  Detail & Related papers  (2025-01-24T11:52:52Z)
• Transformers and Slot Encoding for Sample Efficient Physical World Modelling [1.5498250598583487]
  We propose an architecture combining Transformers for world modelling with the slot-attention paradigm, an approach for learning representations of objects appearing in a scene. We describe the resulting neural architecture and report experimental results showing an improvement over the existing solutions in terms of sample efficiency and a reduction of the variation of the performance over the training examples.
  arXiv  Detail & Related papers  (2024-05-30T15:48:04Z)
• Language-Guided World Models: A Model-Based Approach to AI Control [31.9089380929602]
  This paper introduces the concept of Language-Guided World Models (LWMs) LWMs are probabilistic models that can simulate environments by reading texts. We take initial steps in developing robust LWMs that can generalize to compositionally novel language descriptions.
  arXiv  Detail & Related papers  (2024-01-24T03:11:36Z)
• Transformers are Universal Predictors [21.92580010179886]
  We find limits to the Transformer architecture for language modeling and show it has a universal prediction property in an information-theoretic sense. We analyze performance in non-asymptotic data regimes to understand the role of various components of the Transformer architecture, especially in the context of data-efficient training.
  arXiv  Detail & Related papers  (2023-07-15T16:19:37Z)
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion [66.21290235237808]
  We introduce an energy constrained diffusion model which encodes a batch of instances from a dataset into evolutionary states. We provide rigorous theory that implies closed-form optimal estimates for the pairwise diffusion strength among arbitrary instance pairs. Experiments highlight the wide applicability of our model as a general-purpose encoder backbone with superior performance in various tasks.
  arXiv  Detail & Related papers  (2023-01-23T15:18:54Z)
• Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model [58.17021225930069]
  We explain the rationality of Vision Transformer by analogy with the proven practical Evolutionary Algorithm (EA) We propose a more efficient EAT model, and design task-related heads to deal with different tasks more flexibly. Our approach achieves state-of-the-art results on the ImageNet classification task compared with recent vision transformer works.
  arXiv  Detail & Related papers  (2021-05-31T16:20:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.