Self-Evolving World Models for LLM Agent Planning

• URL: http://arxiv.org/abs/2606.30639v1
• Date: Mon, 29 Jun 2026 17:58:43 GMT
• Title: Self-Evolving World Models for LLM Agent Planning
• Abstract summary: We introduce WorldEvolver, a self-evolving world model framework that revises its deployment-time context while keeping the downstream agent and all model parameters frozen.<n>WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines on downstream agent success rate.
• Score: 72.99782619992361
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World models offer a principled way to equip long-horizon LLM agents with foresight: predictions of action consequences before execution. However, unreliable foresight can be ignored, misused, or even degrade downstream decision-making. In this paper, we introduce WorldEvolver, a self-evolving world model framework that revises its deployment-time context while keeping the downstream agent and all model parameters frozen. WorldEvolver integrates three modules: (i) Episodic Memory, which exploits real action transitions through retrieval-based simulation; (ii) Semantic Memory, which extracts persistent heuristic rules from prediction-observation mismatches; and (iii) Selective Foresight, which filters low-confidence predictions before integrating them into agent reasoning context. We evaluate WorldEvolver on ALFWorld and ScienceWorld, measuring world model prediction accuracy on Word2World and downstream agent success rate on AgentBoard. Extensive experiments show that WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines on downstream agent success rate, demonstrating that test-time memory revision enhances both predictive fidelity and planning performance.

Related papers

• Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning [32.26884129316165]
  Large language model (LLM) agents have demonstrated strong capability in sequential decision-making.<n>Unlike humans who employ "what-if" reasoning to evaluate potential plans before commitment, standard agents lack an internal world model to simulate future outcomes.<n>We propose to internalize future-aware planning by training a single autoregressive model to verbalize both a prospective state rollout and a plan-conditioned success estimate.
  arXiv  Detail & Related papers  (2026-06-25T19:05:44Z)
• COMAP: Co-Evolving World Models and Agent Policies for LLM Agents [14.918267305899619]
  COMAP is a novel framework that co-evolves textual world models and agent policies through closed-loop interaction.<n>At each decision step, the world model predicts future state feedback for candidate actions, and the agent performs future-aware reflection by estimating the reliability of this feedback.<n>The resulting on-policy trajectories are then used to update the world model via self-distillation, allowing it to better match the agent's evolving interaction distribution.
  arXiv  Detail & Related papers  (2026-06-01T15:21:17Z)
• Feedback World Model Enables Precise Guidance of Diffusion Policy [36.965417653906535]
  We propose feedback world model, a new paradigm that closes the loop between prediction and observation at inference time.<n>We show that our method substantially improves both prediction accuracy and policy performance under distribution shift.<n>In particular, it reduces world model prediction error by up to 76.4% and improves out-of-distribution (OOD) success rate by 30%.
  arXiv  Detail & Related papers  (2026-05-15T07:52:13Z)
• HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models [58.191567345416836]
  World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics.<n>HarmoWAM fully leverages a world model to unify predictive and reactive control, enabling both general transit and precise manipulation.<n>We construct three training-unseen test environments across six real-world robotic tasks, covering variations in background, position, and object semantics.
  arXiv  Detail & Related papers  (2026-05-11T17:59:56Z)
• Beyond State Consistency: Behavior Consistency in Text-Based World Models [58.216587360435305]
  We introduce a new behavior-aligned training paradigm aimed at improving the functional consistency between the world model and the real environment.<n> Experiments on WebShop and TextWorld show that BehR-based training improves long-term alignment in several settings.
  arXiv  Detail & Related papers  (2026-04-15T12:56:45Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Current Agents Fail to Leverage World Model as Tool for Foresight [61.82522354207919]
  Generative world models offer a promising remedy: agents could use them to foresee outcomes before acting.<n>This paper empirically examines whether current agents can leverage such world models as tools to enhance their cognition.
  arXiv  Detail & Related papers  (2026-01-07T13:15:23Z)
• Agent2World: Learning to Generate Symbolic World Models via Adaptive Multi-Agent Feedback [51.22403664895878]
  Agent2World is a tool-augmented multi-agent framework that achieves strong inference-time world-model generation.<n>It also serves as a data engine for supervised fine-tuning, by grounding generation in multi-agent feedback.
  arXiv  Detail & Related papers  (2025-12-26T18:54:14Z)
• R-WoM: Retrieval-augmented World Model For Computer-use Agents [15.812606459788471]
  Large Language Models (LLMs) can serve as world models to enhance agent decision-making in digital environments.<n>We probe two core capabilities of world models--future state prediction and reward estimation--through three tasks.<n>We propose the Retrieval-augmented World Model (R-WoM), which grounds simulations by incorporating factual, up-to-date knowledge retrieved from external tutorials.
  arXiv  Detail & Related papers  (2025-10-13T19:52:04Z)
• RLVR-World: Training World Models with Reinforcement Learning [41.04369775904968]
  We present RLVR-World, a unified framework that leverages reinforcement learning with verifiable rewards.<n>We demonstrate substantial performance gains on both language- and video-based world models across domains, including text games, web navigation, and robot manipulation.
  arXiv  Detail & Related papers  (2025-05-20T05:02:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.