From Word to World: Can Large Language Models be Implicit Text-based World Models?

• URL: http://arxiv.org/abs/2512.18832v1
• Date: Sun, 21 Dec 2025 17:28:42 GMT
• Title: From Word to World: Can Large Language Models be Implicit Text-based World Models?
• Abstract summary: Agentic reinforcement learning increasingly relies on experience-driven scaling.<n>World models offer a potential way to improve learning efficiency through simulated experience.<n>We study whether large language models can reliably serve this role and under what conditions they meaningfully benefit agents.
• Score: 82.47317196099907
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic reinforcement learning increasingly relies on experience-driven scaling, yet real-world environments remain non-adaptive, limited in coverage, and difficult to scale. World models offer a potential way to improve learning efficiency through simulated experience, but it remains unclear whether large language models can reliably serve this role and under what conditions they meaningfully benefit agents. We study these questions in text-based environments, which provide a controlled setting to reinterpret language modeling as next-state prediction under interaction. We introduce a three-level framework for evaluating LLM-based world models: (i) fidelity and consistency, (ii) scalability and robustness, and (iii) agent utility. Across five representative environments, we find that sufficiently trained world models maintain coherent latent state, scale predictably with data and model size, and improve agent performance via action verification, synthetic trajectory generation, and warm-starting reinforcement learning. Meanwhile, these gains depend critically on behavioral coverage and environment complexity, delineating clear boundry on when world modeling effectively supports agent learning.

Related papers

• WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling [53.14409745466386]
  World models aim to capture environment dynamics in ways that support perception, reasoning, and action.<n>We introduce textttWorldBagel, a unified VLAW framework built on BAGEL, a modern multimodal unified model.<n>We show that textttWorldBagel consistently outperforms task-specific alternatives and learns action representations that are more structured and semantically aligned with visual and linguistic context.
  arXiv  Detail & Related papers  (2026-07-03T16:18:20Z)
• Bridging the Agent-World Gap: Text World Models for LLM-based Agents [83.8237017249529]
  Large language model (LLM)-based agents are increasingly used in interactive textual environments.<n>Many remain largely reactive, mapping observations to actions without an explicit model of how these environments are structured and evolve.<n>This motivates text world models (TWMs): transition models over textual states that, given a state and a candidate action, predict the resulting webpage, terminal output, API response, or user reply.<n>We systematically review text world models for LLM-based agents, organized around a formal framework and the agent lifecycle.
  arXiv  Detail & Related papers  (2026-06-08T04:58:52Z)
• Learning POMDP World Models from Observations with Language-Model Priors [52.11151769796862]
  Partially-observable Markov decision processes (POMDPs) provide a flexible modeling class for internal world models.<n>We ask whether language-model priors can reduce costly interaction by leveraging prior knowledge.<n>We show that emphPinductor matches the performance and sample efficiency of LLM-based POMDP learning methods.
  arXiv  Detail & Related papers  (2026-05-13T16:18:15Z)
• Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL [0.8546394944138487]
  We formalize text-based world modeling as a steerable transition-dynamics problem decomposed into initial state.<n>We curate 239,403 grounded state-action trajectories spanning nine open-source environments and twelve frontier model families.<n>We introduce a plug-and-play GRPO training framework with deterministic state checks, and perform zero-shot transfer ablations on three OOD environments.
  arXiv  Detail & Related papers  (2026-05-07T00:40:32Z)
• Beyond Language Modeling: An Exploration of Multimodal Pretraining [125.34714978184638]
  We provide empirical clarity through controlled, from-scratch pretraining experiments.<n>We adopt the Transfusion framework, using next-token prediction for language and diffusion for vision.<n>We demonstrate that the MoE architecture harmonizes this scaling asymmetry by providing the high model capacity required by language.
  arXiv  Detail & Related papers  (2026-03-03T18:58:00Z)
• Foundation World Models for Agents that Learn, Verify, and Adapt Reliably Beyond Static Environments [3.282750994783201]
  Next generation of autonomous agents must not only learn efficiently but also act reliably and adapt their behavior in open worlds.<n>This paper outlines a vision for foundation world models: persistent, compositional representations that unify reinforcement learning, reactive/program synthesis, and abstraction mechanisms.<n>We propose an agenda built around four components: (i) learnable reward models from specifications to support optimization with clear objectives; (ii) adaptive formal verification integrated throughout learning; (iii) online abstraction calibration to quantify the reliability of the model's predictions; and (iv) test-time synthesis and world-model generation guided by verifiers.
  arXiv  Detail & Related papers  (2026-02-27T13:20:46Z)
• Reinforcement World Model Learning for LLM-based Agents [60.65003139516272]
  Reinforcement World Model Learning (RWML) is a self-conditioned method that learns action-supervised world models for LLM-based agents.<n>Our method aligns simulated next states produced by the model with realized next states observed from the environment.<n>We evaluate our method on ALFWorld and $2$ Bench and observe significant gains over the base model, despite being entirely self-supervised.
  arXiv  Detail & Related papers  (2026-02-05T16:30:08Z)
• Aligning Agentic World Models via Knowledgeable Experience Learning [68.85843641222186]
  We introduce WorldMind, a framework that constructs a symbolic World Knowledge Repository by synthesizing environmental feedback.<n>WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.
  arXiv  Detail & Related papers  (2026-01-19T17:33:31Z)
• Language-conditioned world model improves policy generalization by reading environmental descriptions [20.07554058793324]
  To interact effectively with humans in the real world, it is important for agents to understand language that describes the dynamics of the environment.<n>We propose a model-based reinforcement learning approach, where a language-conditioned world model is trained through interaction with the environment.<n>We show that policies trained with LED-WM generalize more effectively to unseen games described by novel dynamics and language.
  arXiv  Detail & Related papers  (2025-11-28T06:13:27Z)
• WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Model [55.276852838877346]
  Self-evolving agents are trained on trajectories sampled autonomously based on their own policies.<n>We propose a novel framework that introduces a co-evolving World Model LLM.<n>This world model predicts the next observation based on the current observation and action within the web environment.
  arXiv  Detail & Related papers  (2025-04-23T02:54:31Z)
• AdaWorld: Learning Adaptable World Models with Latent Actions [76.50869178593733]
  We propose AdaWorld, an innovative world model learning approach that enables efficient adaptation.<n>Key idea is to incorporate action information during the pretraining of world models.<n>We then develop an autoregressive world model that conditions on these latent actions.
  arXiv  Detail & Related papers  (2025-03-24T17:58:15Z)
• Making Large Language Models into World Models with Precondition and Effect Knowledge [1.8561812622368763]
  We show that Large Language Models (LLMs) can be induced to perform two critical world model functions. We validate that the precondition and effect knowledge generated by our models aligns with human understanding of world dynamics.
  arXiv  Detail & Related papers  (2024-09-18T19:28:04Z)
• Improving Large Language Model (LLM) fidelity through context-aware grounding: A systematic approach to reliability and veracity [0.0]
  Large Language Models (LLMs) are increasingly sophisticated and ubiquitous in natural language processing (NLP) applications. This paper presents a novel framework for contextual grounding in textual models, with a particular emphasis on the Context Representation stage. Our findings have significant implications for the deployment of LLMs in sensitive domains such as healthcare, legal systems, and social services.
  arXiv  Detail & Related papers  (2024-08-07T18:12:02Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.