A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens

• URL: http://arxiv.org/abs/2604.04913v1
• Date: Mon, 06 Apr 2026 17:55:05 GMT
• Title: A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens
• Abstract summary: We introduce DeltaTok, a tokenizer that encodes the feature difference between consecutive frames into a single "delta" token.<n>We also present DeltaWorld, a generative world model operating on these tokens to efficiently generate diverse plausible futures.
• Score: 40.71386517238762
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Anticipating diverse future states is a central challenge in video world modeling. Discriminative world models produce a deterministic prediction that implicitly averages over possible futures, while existing generative world models remain computationally expensive. Recent work demonstrates that predicting the future in the feature space of a vision foundation model (VFM), rather than a latent space optimized for pixel reconstruction, requires significantly fewer world model parameters. However, most such approaches remain discriminative. In this work, we introduce DeltaTok, a tokenizer that encodes the VFM feature difference between consecutive frames into a single continuous "delta" token, and DeltaWorld, a generative world model operating on these tokens to efficiently generate diverse plausible futures. Delta tokens reduce video from a three-dimensional spatio-temporal representation to a one-dimensional temporal sequence, for example yielding a 1,024x token reduction with 512x512 frames. This compact representation enables tractable multi-hypothesis training, where many futures are generated in parallel and only the best is supervised. At inference, this leads to diverse predictions in a single forward pass. Experiments on dense forecasting tasks demonstrate that DeltaWorld forecasts futures that more closely align with real-world outcomes, while having over 35x fewer parameters and using 2,000x fewer FLOPs than existing generative world models. Code and weights: https://deltatok.github.io.

Related papers

• X-Mind: Efficient Visual Chain-of-Thought via Predictive World Model for End-to-End Driving [40.275378373130245]
  X-Mind is a framework that embeds forward-looking reasoning into autonomous driving policies.<n>By enforcing a world rollout prior to action, the model is constrained to imagine future evolution first.<n>X-Mind achieves competitive end-to-end driving performance on resource-constrained vehicle platforms.
  arXiv  Detail & Related papers  (2026-06-27T06:17:50Z)
• Geometric Action Model for Robot Policy Learning [68.6657929619782]
  Generalist robot policies must follow user instructions while reasoning about how objects, cameras, and robot actions interact in the 3D physical world.<n>Recent vision-language-action models (VLAs) and video world-action models (WAMs) inherit strong semantic or temporal priors from large-scale foundation models.<n>We propose the Geometric Action Model (GAM), a language-conditioned manipulation policy.
  arXiv  Detail & Related papers  (2026-06-15T17:58:03Z)
• FluidWorld: Reaction-Diffusion Dynamics as a Predictive Substrate for World Models [0.0]
  Current approaches default to Transformer-based predictors operating in learned latent spaces.<n>This paper asks a foundational question: is self-attention necessary for predictive world modeling, or can alternative computational substrates achieve comparable or superior results?<n>I introduce FluidWorld, a proof-of-concept world model whose predictive dynamics are governed by partial differential equations (PDEs) of reaction-diffusion type.
  arXiv  Detail & Related papers  (2026-03-22T16:49:43Z)
• Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens [88.42820935044021]
  We presentCub Discrete Diffusion (CubiD), the first discrete generation model for high-dimensional representations.<n>CubiD performs fine-grained masking throughout the high-dimensional discrete representation.<n>On ImageNet-256, CubiD achieves state-of-the-art discrete generation with strong scaling behavior from 900M to 3.7B parameters.
  arXiv  Detail & Related papers  (2026-03-19T17:59:55Z)
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels [49.35636088613484]
  Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces.<n>We introduce LeModelWorld, the first JEPA that trains stably end-to-end from raw pixels using only two loss terms.<n>With 15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models.
  arXiv  Detail & Related papers  (2026-03-13T19:48:14Z)
• Delta-Triplane Transformers as Occupancy World Models [57.16979927973973]
  Occupancy World Models (OWMs) aim to predict future scenes via 3D voxelized representations of the environment to support intelligent motion planning.<n>We propose Delta-Triplane Transformers (DTT), a novel 4D OWM for autonomous driving, that introduces two key innovations.<n>DTT delivers a 1.44$times$ speedup (26 FPS) over the state of the art, improves mean IoU to 30.85, and reduces the mean absolute planning error to 1.0 meters.
  arXiv  Detail & Related papers  (2025-03-10T13:50:23Z)
• FIRP: Faster LLM inference via future intermediate representation prediction [54.897493351694195]
  FIRP generates multiple tokens instead of one at each decoding step. We conduct extensive experiments, showing a speedup ratio of 1.9x-3x in several models and datasets.
  arXiv  Detail & Related papers  (2024-10-27T15:53:49Z)
• Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens [53.99177152562075]
  Scaling up autoregressive models in vision has not proven as beneficial as in large language models. We focus on two critical factors: whether models use discrete or continuous tokens, and whether tokens are generated in a random or fixed order using BERT- or GPT-like transformer architectures. Our results show that while all models scale effectively in terms of validation loss, their evaluation performance -- measured by FID, GenEval score, and visual quality -- follows different trends.
  arXiv  Detail & Related papers  (2024-10-17T17:59:59Z)
• Efficient World Models with Context-Aware Tokenization [22.84676306124071]
  $Delta$-IRIS is a new agent with a world model architecture composed of a discrete autoencoder that encodes deltas between time steps. In the Crafter benchmark, $Delta$-IRIS sets a new state of the art at multiple frame budgets, while being an order of magnitude faster to train than previous attention-based approaches.
  arXiv  Detail & Related papers  (2024-06-27T16:54:12Z)
• Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion [36.321494200830244]
  Copilot4D is a novel world modeling approach that first tokenizes sensor observations with VQVAE, then predicts the future via discrete diffusion. Our results demonstrate that discrete diffusion on tokenized agent experience can unlock the power of GPT-like unsupervised learning for robotics.
  arXiv  Detail & Related papers  (2023-11-02T06:21:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.