World models of environment, agent and joint agent-environment systems

• URL: http://arxiv.org/abs/2608.20401v1
• Date: Thu, 23 Jul 2026 13:35:03 GMT
• Title: World models of environment, agent and joint agent-environment systems
• Abstract summary: We argue that there is a prior distinction: which channel they model.<n>We consider three cases: the environment channel $O_: mid A_:$, the agent channel $A_: mid O_:$, and the realised joint process $(A, O)_:$, equivalently viewed as a channel with no inputs.
• Score: 76.81951414842736
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models are a central component of model-based reinforcement learning. They are usually discussed in terms of what variables they predict, such as observations, rewards, states, latent or information states. We argue that there is a prior distinction: which channel they model. We consider three cases: the environment channel $O_{:} \mid A_{:}$, the agent channel $A_{:} \mid O_{:}$, and the realised joint process $(A, O)_{:}$, equivalently viewed as a channel with no inputs. Using computational mechanics, we define canonical predictive models for these three cases as $ε$-transducers or $ε$-machines. Canonical environment models recover standard predictive state representations, while the other two give analogous notions of canonical models for the agent and the joint system. We then build canonical support-restricted environment and agent models induced by closed-loop coupling, whose predictive equivalences range over continuations supported by the realised interaction. The key structural result is that canonical support-restricted environment states factor through the canonical joint causal states, and their transition structure is induced directly from the joint model; the agent-side construction is dual. Finally, we give a POMDP/controller example in which the unrestricted environment model has infinitely many states while the canonical support-restricted model induced by the coupling is finite. The framework clarifies what different world models are models of, and how coupling and support restriction can change their canonical predictive structure and complexity.

Related papers

• Inverting the Bellman Equation: From $Q$-Values to World Models [57.827849584133425]
  We prove that value-based agents trained on a sufficiently rich set of reward functions implicitly encode a unique and accurate world model.<n>We find that policies trained exclusively on a ttReacher agent's implicit world model are quasi-optimal on out-of-distribution, velocity-based goals despite position-only training.
  arXiv  Detail & Related papers  (2026-06-19T07:26:14Z)
• Collective Alignment in LLM Multi-Agent Systems: Disentangling Bias from Cooperation via Statistical Physics [0.0]
  We present a model-agnostic statistical-physics method to disentangle social conformity from intrinsic bias, compute critical exponents, and probe the collective behavior and possible phase transitions of multi-agent systems.<n>We find that collective alignment is dominated by an intrinsic bias rather than by cooperative neighbor coupling, producing field-driven crossovers instead of genuine phase transitions.
  arXiv  Detail & Related papers  (2026-05-11T13:13:44Z)
• FluidWorld: Reaction-Diffusion Dynamics as a Predictive Substrate for World Models [0.0]
  Current approaches default to Transformer-based predictors operating in learned latent spaces.<n>This paper asks a foundational question: is self-attention necessary for predictive world modeling, or can alternative computational substrates achieve comparable or superior results?<n>I introduce FluidWorld, a proof-of-concept world model whose predictive dynamics are governed by partial differential equations (PDEs) of reaction-diffusion type.
  arXiv  Detail & Related papers  (2026-03-22T16:49:43Z)
• MBD: A Model-Based Debiasing Framework Across User, Content, and Model Dimensions [50.00784452900918]
  We propose a general, model-based debiasing (MBD) framework that addresses this challenge.<n>We explicitly estimate the contextual mean and variance of the engagement distribution for arbitrary cohorts.<n>This integration allows the framework to convert biased raw signals into unbiased representations.
  arXiv  Detail & Related papers  (2026-03-15T15:07:01Z)
• $α$-Flow: A Unified Framework for Continuous-State Discrete Flow Matching Models [8.705749038874137]
  This work presents a unified framework for Continuous-State Discrete Flow Matching models.<n>We introduce $alpha$-Flow, a family of CS-DFM models that adheres to the canonical $alpha$-geometry of the statistical manifold.<n>We show that the flow matching loss for $alpha$-flow establishes a unified variational bound for the discrete negative log-likelihood.
  arXiv  Detail & Related papers  (2025-04-14T14:51:45Z)
• Super-model ecosystem: A domain-adaptation perspective [101.76769818069072]
  This paper attempts to establish the theoretical foundation for the emerging super-model paradigm via domain adaptation. Super-model paradigms help reduce computational and data cost and carbon emission, which is critical to AI industry.
  arXiv  Detail & Related papers  (2022-08-30T09:09:43Z)
• On the Generative Utility of Cyclic Conditionals [103.1624347008042]
  We study whether and how can we model a joint distribution $p(x,z)$ using two conditional models $p(x|z)$ that form a cycle. We propose the CyGen framework for cyclic-conditional generative modeling, including methods to enforce compatibility and use the determined distribution to fit and generate data.
  arXiv  Detail & Related papers  (2021-06-30T10:23:45Z)
• Generative Temporal Difference Learning for Infinite-Horizon Prediction [101.59882753763888]
  We introduce the $gamma$-model, a predictive model of environment dynamics with an infinite probabilistic horizon. We discuss how its training reflects an inescapable tradeoff between training-time and testing-time compounding errors.
  arXiv  Detail & Related papers  (2020-10-27T17:54:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.