PatchWorld: Gradient-Free Optimization of Executable World Models

• URL: http://arxiv.org/abs/2605.30880v1
• Date: Fri, 29 May 2026 06:11:14 GMT
• Title: PatchWorld: Gradient-Free Optimization of Executable World Models
• Abstract summary: We introduce PatchWorld, a gradient-free framework that turns offline trajectories into executable Python world models.<n>Across seven AgentGym environments, PatchWorld-Simple achieves the highest code-based planning score.<n>We find that a human-specified residual-memory bias improves surface observation fidelity but weakens decision utility.
• Score: 69.05094454437402
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-agent environments are typically modeled as partially observable Markov decision processes (POMDPs), assuming that the simulator's latent state and transition dynamics are hidden from the agent. Yet little work has examined whether executable code can be induced to serve as a world model for prediction and planning under partial observability. We introduce PatchWorld, a gradient-free framework that turns offline trajectories into executable Python world models through counterexample-guided code repair. Instead of predicting the next observation with a black-box model, PatchWorld induces symbolic belief-state programs whose action updates can be inspected, replayed, and locally patched. Across seven AgentGym environments, PatchWorld-Simple achieves the highest code-based planning score among evaluated methods, reaching 76.4\% macro success in live one-step lookahead while invoking no LLM calls inside the world-model prediction module itself. We further find that a human-specified residual-memory bias improves surface observation fidelity but weakens decision utility. This exposes a tradeoff in executable world models, since improving observation fidelity can come at the expense of action-discriminative dynamics, and vice versa. Code is available at https://github.com/HKBU-KnowComp/PatchWorld.

Related papers

• Discriminative World Models for Web Agents [52.31107486410659]
  Recent web agents use world models for test-time action selection by sampling candidate actions, predicting the resulting web states, and ranking them with a ranker model or a Process Reward Model (PRM)<n>We introduce predicted-state matching, a training objective where the predicted representation must distinguish the true resulting state from those reached by alternative actions.<n>Our approach improves PRM-style action ranking on WebPRMBench compared with action-only PRMs and PRMs augmented with supervised-next-state world models.
  arXiv  Detail & Related papers  (2026-09-02T17:59:40Z)
• VisualPatchWorld: Code World Models as Latent Structured Representations for Planning [8.39933163242427]
  We introduce VisualPatchWorld (VPW), which represents world dynamics as code.<n>VPW first selects a qualitative dynamical form with short active probes, then fits that form's free parameters from recorded state-action traces.<n>VPW attains 69.0% mean planning success and exceeds the strongest code baseline by 23.5 points.
  arXiv  Detail & Related papers  (2026-07-28T03:23:47Z)
• Self-Evolving World Models for LLM Agent Planning [72.99782619992361]
  We introduce WorldEvolver, a self-evolving world model framework that revises its deployment-time context while keeping the downstream agent and all model parameters frozen.<n>WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines on downstream agent success rate.
  arXiv  Detail & Related papers  (2026-06-29T17:58:43Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• Mind-Studio: Executable World Models with Lookahead Evaluation for Partially Observable Games [26.359796001522707]
  Mind-Studio is a framework that synthesizes pygame-style world models from state-action-next-state trajectories.<n>On Montezuma's Revenge, Mind-Studio improves chosen-action next-state prediction from 0.3% for PoE-World to 48.7% while verifying 5 of 8 subgoals.
  arXiv  Detail & Related papers  (2026-06-14T23:53:49Z)
• Beyond State Consistency: Behavior Consistency in Text-Based World Models [58.216587360435305]
  We introduce a new behavior-aligned training paradigm aimed at improving the functional consistency between the world model and the real environment.<n> Experiments on WebShop and TextWorld show that BehR-based training improves long-term alignment in several settings.
  arXiv  Detail & Related papers  (2026-04-15T12:56:45Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Parallel Stochastic Gradient-Based Planning for World Models [39.699893143984916]
  We propose a robust and highly parallelizable planner that leverages the differentiability of the learned world model for efficient optimization.<n>Our method treats states as optimization variables ("virtual states") with soft dynamics constraints, enabling parallel and easier optimization.<n>Our planner, which we call GRASP (GradAxed Planner), can be viewed as a valid version of a non-condensed or collocation-based optimal controller.
  arXiv  Detail & Related papers  (2026-01-31T02:57:47Z)
• WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning [52.36434784963598]
  We introduce WorldPrediction, a video-based benchmark for evaluating world modeling and procedural planning capabilities of different AI models.<n>We show that current frontier models barely achieve 57% accuracy on WorldPrediction-WM and 38% on WorldPrediction-PP whereas humans are able to solve both tasks perfectly.
  arXiv  Detail & Related papers  (2025-06-04T18:22:40Z)
• RLVR-World: Training World Models with Reinforcement Learning [41.04369775904968]
  We present RLVR-World, a unified framework that leverages reinforcement learning with verifiable rewards.<n>We demonstrate substantial performance gains on both language- and video-based world models across domains, including text games, web navigation, and robot manipulation.
  arXiv  Detail & Related papers  (2025-05-20T05:02:53Z)
• AI in a vat: Fundamental limits of efficient world modelling for agent sandboxing and interpretability [84.52205243353761]
  Recent work proposes using world models to generate controlled virtual environments in which AI agents can be tested before deployment.<n>We investigate ways of simplifying world models that remain agnostic to the AI agent under evaluation.
  arXiv  Detail & Related papers  (2025-04-06T20:35:44Z)
• Learning World Models for Unconstrained Goal Navigation [4.549550797148707]
  We introduce a goal-directed exploration algorithm, MUN, for learning world models. MUN is capable of modeling state transitions between arbitrary subgoal states in the replay buffer. Results demonstrate that MUN strengthens the reliability of world models and significantly improves the policy's capacity to generalize.
  arXiv  Detail & Related papers  (2024-11-03T01:35:06Z)
• Defensive Patches for Robust Recognition in the Physical World [111.46724655123813]
  Data-end defense improves robustness by operations on input data instead of modifying models. Previous data-end defenses show low generalization against diverse noises and weak transferability across multiple models. We propose a defensive patch generation framework to address these problems by helping models better exploit these features.
  arXiv  Detail & Related papers  (2022-04-13T07:34:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.