Discriminative World Models for Web Agents

• URL: http://arxiv.org/abs/2609.02885v1
• Date: Wed, 02 Sep 2026 17:59:40 GMT
• Title: Discriminative World Models for Web Agents
• Abstract summary: Recent web agents use world models for test-time action selection by sampling candidate actions, predicting the resulting web states, and ranking them with a ranker model or a Process Reward Model (PRM)<n>We introduce predicted-state matching, a training objective where the predicted representation must distinguish the true resulting state from those reached by alternative actions.<n>Our approach improves PRM-style action ranking on WebPRMBench compared with action-only PRMs and PRMs augmented with supervised-next-state world models.
• Score: 52.31107486410659
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent web agents use world models for test-time action selection by sampling candidate actions, predicting the resulting web states, and ranking them with a ranker model or a Process Reward Model (PRM). These world models are typically trained via supervised next-state prediction to generate fixed representations like HTML or AXTree snapshots. However, this objective is misaligned with the downstream ranker, which relies on predicted states being discriminative across candidates to accurately score them. To address this, we introduce predicted-state matching, a training objective where the predicted representation must distinguish the true resulting state from those reached by alternative actions. We train these models using a branching web-agent dataset derived from WebArena Go-Browse trajectories, where every decision point contains multiple alternative actions and their resulting states. Experiments on our held-out predicted-state matching benchmark show that our approach outperforms world models trained with supervised next-state prediction. We further show that our approach improves PRM-style action ranking on WebPRMBench compared with action-only PRMs and PRMs augmented with supervised-next-state world models. Finally, on WebArena-Lite, using our world model for test-time action selection improves end-to-end task success. Our project page is available at: https://dhruvpendharkar.github.io/dwm/.

Related papers

• World-Coherent Decoding: Self-Verifying Test-Time Planning for World Action Models [20.97725301919032]
  World-Coherent-Decoding (WCD) is a self-verifying test-time planning framework for WAMs.<n>WCD trains a lightweight online predictor for future candidate selection.<n>On RoboTwin 2.0, WCD improves Hard success under limited randomized-scene supervision from $55.80%$ to $60.90%$, with a $+16.43$ gains on Horizon-3 tasks.<n>These results highlight a simple principle: test-time scaling for WAMs depends less on sampling more futures than on selecting reliable ones.
  arXiv  Detail & Related papers  (2026-09-02T06:21:36Z)
• Impression Share Prediction: An Offline Evaluation Task for Ranking Systems [2.455009405787869]
  We propose emphimpression share prediction as an offline evaluation task.<n>The task is inherently counterfactual, since the candidate has never served live traffic.<n>We develop a statistical learning framework that predicts impression shares from a candidate's early-interaction confidence signals.
  arXiv  Detail & Related papers  (2026-08-17T17:52:55Z)
• Self-Evolving World Models for LLM Agent Planning [72.99782619992361]
  We introduce WorldEvolver, a self-evolving world model framework that revises its deployment-time context while keeping the downstream agent and all model parameters frozen.<n>WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines on downstream agent success rate.
  arXiv  Detail & Related papers  (2026-06-29T17:58:43Z)
• COMAP: Co-Evolving World Models and Agent Policies for LLM Agents [14.918267305899619]
  COMAP is a novel framework that co-evolves textual world models and agent policies through closed-loop interaction.<n>At each decision step, the world model predicts future state feedback for candidate actions, and the agent performs future-aware reflection by estimating the reliability of this feedback.<n>The resulting on-policy trajectories are then used to update the world model via self-distillation, allowing it to better match the agent's evolving interaction distribution.
  arXiv  Detail & Related papers  (2026-06-01T15:21:17Z)
• TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulation [61.35569005726248]
  Existing embodied control research demonstrates remarkable performance improvements by scaling training data and model size.<n>Non-deterministic generative models, such as diffusion and autoregressive models, have been widely adopted in the field of embodied control.<n>We propose textbfTapSampling, a plug-and-play framework for inference-time sampling.
  arXiv  Detail & Related papers  (2026-05-25T08:03:31Z)
• Learning Lifted Action Models from Unsupervised Visual Traces [38.479081137300945]
  We propose a deep learning framework that jointly learns state prediction, action prediction, and a lifted action model.<n>We also introduce a mixed-integer linear program (MILP) to prevent prediction collapse and self-reinforcing errors.<n>Experiments across multiple domains show that integrating MILP-based correction helps the model escape local optima and converge toward globally consistent solutions.
  arXiv  Detail & Related papers  (2026-04-21T03:49:04Z)
• Beyond State Consistency: Behavior Consistency in Text-Based World Models [58.216587360435305]
  We introduce a new behavior-aligned training paradigm aimed at improving the functional consistency between the world model and the real environment.<n> Experiments on WebShop and TextWorld show that BehR-based training improves long-term alignment in several settings.
  arXiv  Detail & Related papers  (2026-04-15T12:56:45Z)
• Benchmarking World-Model Learning [48.64929865461133]
  We present WorldTest, a protocol to evaluate model-learning agents that separates a scored phase in a different but related environment.<n>We compared 517 human participants and three models on AutumnBench.<n>We found that humans outperform the models in some environments but not others.
  arXiv  Detail & Related papers  (2025-10-22T17:23:18Z)
• WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning [52.36434784963598]
  We introduce WorldPrediction, a video-based benchmark for evaluating world modeling and procedural planning capabilities of different AI models.<n>We show that current frontier models barely achieve 57% accuracy on WorldPrediction-WM and 38% on WorldPrediction-PP whereas humans are able to solve both tasks perfectly.
  arXiv  Detail & Related papers  (2025-06-04T18:22:40Z)
• A Control-Centric Benchmark for Video Prediction [69.22614362800692]
  We propose a benchmark for action-conditioned video prediction in the form of a control benchmark. Our benchmark includes simulated environments with 11 task categories and 310 task instance definitions. We then leverage our benchmark to study the effects of scaling model size, quantity of training data, and model ensembling.
  arXiv  Detail & Related papers  (2023-04-26T17:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.