One Lens, Many Worlds : A Capability-Typed Interface for World-Model Interpretability

• URL: http://arxiv.org/abs/2606.09936v1
• Date: Sun, 07 Jun 2026 19:27:04 GMT
• Title: One Lens, Many Worlds : A Capability-Typed Interface for World-Model Interpretability
• Abstract summary: We present WorldModelLens, an open-source interpretability substrate organized around a capability-typed adapter.<n>Every model implements four required methods (encode, transition, initial state, sample) and declares a set of optional heads.<n>A single hook and cache layer exposes time-indexed activations, imagination rollouts, and intervention replay over this interface.
• Score: 2.550140611921313
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: World models are now built on substantially different computational substrates. Latent recurrent state-space models such as PlaNet and the Dreamer family compress observations into recurrent states; token-based models such as IRIS quantize observations into a learned codebook and predict autoregressively with a transformer; and joint-embedding predictive architectures such as I-JEPA predict in a learned latent space with no pixel decoder. The interpretability methods applied to these models, including probing, activation patching, sparse autoencoders, and surprise analysis, share a common set of primitives, yet they are re-implemented from scratch for each architecture because existing hook-and-cache tooling assumes a transformer language model with no notion of actions, environment steps, or imagined rollouts. We argue that this fragmentation reflects the tooling rather than the models, and that the shared structure of world models is captured by a small typed interface. We present WorldModelLens, an open-source interpretability substrate organized around a capability-typed adapter: every model implements four required methods (encode, transition, initial state, sample) and declares a set of optional heads (decode, reward, continue, actor, critic) through an explicit capability descriptor, so that reinforcement-learning and self-supervised world models are first-class without either imitating the other. A single hook and cache layer exposes time-indexed activations, imagination rollouts, and intervention replay over this interface, allowing each analysis to be written once.

Related papers

• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modeling [51.40150411616207]
  We introduce Latent Particle World Model (LPWM), a self-supervised object-centric world model scaled to real-world multi-object datasets.<n>LPWM autonomously discovers keypoints, bounding boxes, and object masks directly from video data.<n>Our architecture is trained end-to-end purely from videos and supports flexible conditioning on actions, language, and image goals.
  arXiv  Detail & Related papers  (2026-03-04T19:36:08Z)
• A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures [58.26804959656713]
  We present EB-JEPA, an open-source library for learning representations and world models using Joint-Embedding Predictive Architectures (JEPAs)<n>JEPAs learn to predict in representation space rather than pixel space, avoiding the pitfalls of generative modeling.<n>We show how these representations can drive action-conditioned world models, achieving a 97% planning success rate on the Two Rooms navigation task.
  arXiv  Detail & Related papers  (2026-02-03T14:56:24Z)
• Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment [6.614005142754584]
  Universal Sparse Autoencoders (USAEs) are a framework for uncovering and aligning interpretable concepts spanning multiple deep neural networks.<n>USAEs learn a universal concept space that can reconstruct and interpret the internal activations of multiple models at once.
  arXiv  Detail & Related papers  (2025-02-06T02:06:16Z)
• Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines [74.42485647685272]
  We focus on Generative Masked Language Models (GMLMs) We train a model to fit conditional probabilities of the data distribution via masking, which are subsequently used as inputs to a Markov Chain to draw samples from the model. We adapt the T5 model for iteratively-refined parallel decoding, achieving 2-3x speedup in machine translation with minimal sacrifice in quality.
  arXiv  Detail & Related papers  (2024-07-22T18:00:00Z)
• Slot Structured World Models [0.0]
  State-of-the-art approaches use a feedforward encoder to extract object embeddings and a latent graph neural network to model the interaction between these object embeddings. We introduce it Slot Structured World Models (SSWM), a class of world models that combines an it object-centric encoder with a latent graph-based dynamics model.
  arXiv  Detail & Related papers  (2024-01-08T21:19:30Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Decoupling Global and Local Representations via Invertible Generative Flows [47.366299240738094]
  Experimental results on standard image benchmarks demonstrate the effectiveness of our model in terms of density estimation, image generation and unsupervised representation learning. This work demonstrates that a generative model with a likelihood-based objective is capable of learning decoupled representations, requiring no explicit supervision.
  arXiv  Detail & Related papers  (2020-04-12T03:18:13Z)
• CAZSL: Zero-Shot Regression for Pushing Models by Generalizing Through Context [13.217582954907234]
  We study the problem of designing deep learning agents which can generalize their models of the physical world by building context-aware models. We present context-aware zero shot learning (CAZSL, pronounced as casual) models, an approach utilizing a Siamese network, embedding space and regularization based on context variables. We test our proposed learning algorithm on the recently released Omnipush datatset that allows testing of meta-learning capabilities.
  arXiv  Detail & Related papers  (2020-03-26T01:21:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.